Product Features
Blog
Contact Us
Sign In
Be IT Resourcing Ltd
,
Glasgow, City of Glasgow
Big Data Engineer - Glasgow
Overview
Big Data Engineer Glasgow Do you want to work on ground breaking BI and Artificial Intelligence projects for a leading global name, whilst working with high velocity data from some of the worlds leading retailers? Our client is looking to bring someone into their team to design, implement and optimise the processes by which they ingest data from a variety of sources, including IOT Devices, using Big Data. You will identify and implement optimum data lake storage format and retrieval methods and technologies as well as providing wider support the a team of Data Scientists and BI developers. Design and implement pipelines to ingest data into and consume data from the data lake Source and transform data from historical and streaming data sources Implement automated monitoring and alerting on pipelines to ensure consistent processing of data Design systems to allow real time analytics on high velocity data Proactively ensure optimum performance by revisiting design choices to check they are still fit for purpose Collaborate with Data Scientists and BI Developers to ensure the wider team’s objectives are being met by data lake design Support wider team in consuming data from the data lake Be able to provide technical leadership in the use and design of big data technologies To be successful in this role you should ideally have solid experience with Azure Data Lakes, Azure Data Factory, Databricks and Python/PySpark or Scala. It would also be useful if you had exposure to IOT technology, NOSQL, Azure Data Warehouse and CosmosDB. If this sounds like it could be the role for you then please get in touch with Christina Hall at This job was originally posted as www.totaljobs.com/job/89898027