Product Features
Blog
Contact Us
Sign In
Method Resourcing
,
London, Greater London
Senior Data Engineer- SPARK/SCALA
|
|
Job Type: Full-time |
|---|---|
|
|
Salary: £75,000 per annum |
Overview
Role- Data Engineer Location- Farringdon Salary- up to £75,000- £100,000 Do you want to work with a truly revolutionary technology company? With a company that is taking marketing analytics to the next level, using AI and Machine learning technology? Method Resourcing are working closely with an Analysis tool provider to find a senior data engineer to join there world class team. Having just received over one million pounds in funding they are scaling the data teams, the work will be greenfield work with complex data sets taken from a billions of data touch points daily. They are willing to develop the right person, but you should have more than 2 years commercial developing experience with Scala and Spark What will I be doing? Designing robust spark processes and produce effect data processes, aggregations and pipelines in spark. Designing low latency API's and processes. Monitoring data collection with Java and Scala. Take ownership of your work and invest in the development of our first class product Have a good grasp of modern data practises and apply them to complex problems. Building services features and libraries that contribution to library code and core services Have a good understanding of data architecture Write clear, high quality testable code by working to TDD and SOLID methods. Keys Skills Bachelor degree in Computer Science, similar technical field of study or equivalent practical experience Commercial experience developing Spark jobs using Scala Commercial experience with big data using java/scala Experience in data processing using traditional and distributed systems, Hadoop, Spark, AWS - S3 Experience designing data models and data warehouses Experience in SQL, NoSQL database management systems we use PostgreSQL and Cassandra Nice to haves Commercial messaging technologies experience i.e Kafka or similar Orchestration software - Chef, Puppet, Ansible, Salt Confident with building complex ETL workflows using Luigi or Airflow Good knowledge of AWS or cloud technologies ELK stack experience would be super. Interviews are being held next week, apply to do for your chance to interview. Interview process will be 2 stages, including a short technical test around Scala/Spark. For more information, get in touch with Imogen Moody at Method Resourcing on /