Product Features
Blog
Contact Us
Sign In
Dynamix Recruitment Limited
,
Ipswich, Suffolk
Data Engineer
|
|
Job Type: Full-time |
|---|---|
|
|
Salary: £35,000 per annum |
Overview
As a data engineer within the exciting, new claims advanced analytics capability, you will be building big data solutions to solve some of the organization’s toughest problems and delivering significant business value. This is a really exciting time to join as you will be helping to shape the big data analytics architecture and technology stack within a new cloud based data lake Responsibilities : Shape the portfolio of business problems to solve by building detailed knowledge of data sources (internal and external) Model data landscape, obtain data extracts and define secure data exchange approaches Acquire, ingest, and process data from multiple sources and systems into Cloud Data Lake Operate in fast-paced, iterative environment while remaining compliant with Information Sec policies/standards Collaborate with data scientists to map data fields to hypotheses and curate, wrangle, and prepare data for use in their advanced analytical models Help architect the strategic advanced analytics technology landscape Build re-usable code and data assets Codify best practices, methodology and share knowledge with other data engineers/scientists in the organisation Measures : Become expert in claims data sources Framework set up across the company to define best practice in data engineering space Robust data sources in the data lake with increasing proportion of data held in the lake No unexpected issues arise Successful delivery of cloud projects "Single version of the truth" tables and views in the cloud that are used by a wide variety of end users providing accurate re-producible Skills & Experience : Meaningful experience (2 years) with at least two of the following technologies: Python, Scala, SQL, Java Experience and interest in Cloud platforms such as:, Azure, AWS or Databricks The ability to work across structured, semi-structured, and unstructured data, extracting information and identifying linkages across disparate data sets Meaningful experience in at least one database technology such as: -Distributed Processing (Spark, Hadoop, EMR) -Traditional RDBMS (MS SQL Server, Oracle, MySQL, PostgreSQL) -MPP (AWSRedshift, Teradata) -NoSQL (MongoDB, DynamoDB, Cassandra, Neo4J, Titan) Understanding of Information Security principles to ensure compliant handling and management of data Experience in traditional data warehousing / ETL tools (Informatica, Talend, Pentaho, DataStage) Ability to clearly communicate complex solutions