BigData Developer

Project Description:The Data Engineering position is for a New York based program that sits within the Americas Data Office (ADO). Responsibilities include design, implementation and testing of standard enterprise-wide platforms to capture data assets including analysis and engineering of data represented through these platforms. You will be a member of a team responsible for creation of cloud-based production quality tools used in data governance - enabling the bank to better understand its data and meet regulatory requirements. This requires an ability to analyze large datasets from disparate sources and engineer solutions. Responsibilities:• Build out data trace solution to access, acquire, store, analyse and visualize multiple data sets on Enterprise Analytics Platform.• Baseline lineage collation - System flows, attribute level lineage, filters, manual process transformations and data controls.• Risk Assessment - review Baseline lineage in conjunction with known issues to design Data trace activities.• Execute Data analysis for data completeness, transformation, filters and data control validation.• Data Governance - collaborate with Finance, Risk and Treasury representatives and data stewards on capture of data assets for key outcomes. Mandatory Skills Description:• SQL - SqlServer, Oracle• Big data platform experience - Hive, Hue, Impala, HDFS• Google Cloud Platform - Big Query, Dataproc• Python - pyspark, pandas• Data science skills• Experience with data analytics• Workflow - Pentaho, Oozie• Tools and environments - Apache Spark, Hadoop, CDSW, Anaconda Nice-to-Have Skills:• Good to have: Graph - Virtuoso• Good to have: Programing languages - Python, Scala, Java, Visual Basic• Knowledge of Google Cloud Platform a plus