Big Data Engineer

Ateeca ,
London, Greater London
More Details

Overview

Job Description

Data Engineer to work on build of data pipelines on HDP data platform working on project to ingest XML files to data platform. The data is Securities Financing Transactions data provided by external Trade Repository organisations. The stored data will be surfaced for end user analytical and reporting requirements Development Skills * Experience of delivering Data Pipelines on Hortonworks / Cloudera installations * Experience in Python and Spark and Hive * Data modelling * Distributed computing * Good understanding in best practices for use of source control, preferably with experience of GIT and TFS * Knowledge of industry wide analytical and visualisation tools (Tableau and R) * Linux Skills Need to be Active SC Cleared Additional Information All your information will be kept confidential according to EEO guidelines. Responsibilities: Data Engineer to work on build of data pipelines on HDP data platform working on project to ingest XML files to data platform. The data is Securities Financing Transactions data provided by external Trade Repository organisations. The stored data will be surfaced for end user analytical and reporting requirements Development Skills Experience of delivering Data Pipelines on Hortonworks / Cloudera installations Experience in Python and Spark and Hive Data modelling Distributed computing Good understanding in best practices for use of source control, preferably with experience of GIT and TFS Knowledge of industry wide analytical and visualisation tools (Tableau and R) Linux Skills Need to be Active SC Cleared

People also viewed