Product Features
Blog
Contact Us
Sign In
Harnham
,
London, Greater London
Big Data Engineer - Scala
|
|
Salary: £550 per day |
|---|
Overview
Big Data Engineer - Scala £600 per day 3-months Central London As a Big Data Engineer, you will be migrating away from a Redshift data warehouse and introducing Spark using Airflow for orchestration. This will be for a food-based E-commerce company. THE COMPANY: This company focus on using data to drive their success. They are an E-commerce company who supply food and beverages across the UK. They are using Data Science algorithms to improve efficiency and reduce waste. They have established a data platform that allows Data Scientists to build their models on, however, they are not ready to use big data just yet. This is where you are required to come and make their platform, big data ready. THE ROLE: You will be working alongside Data Scientists to ensure they have the relevant data for them to build their models. You are required to have expertise building ETL pipelines in Python. Ideally, you will have worked with big data using Spark applications and should be familiar with using Airflow. They have an AWS environment but are moving away from Redshift. YOUR SKILLS AND EXPERIENCE: The ideal Big Data Engineer will have: A good understanding of AWS components: Redshift, S3, Lambdas, EC2 Strong coding skills in Python Experience using big data: Spark Worked with orchestration tools like Airflow HOW TO APPLY: Please register your interest by sending your CV via the Apply link on this page.