Azure Big Data Engineer

EPAM Systems Ltd ,
London, Greater London
More Details

Overview

Named as one of Fortune’s List of 100 Fastest Growing Companies for 2019, EPAM is committed to providing our global team of 36,700 EPAMers with inspiring careers from day one. EPAMers lead with passion and honesty, and think creatively. Our people are the source of our success and we value collaboration, try to always understand our customers’ business, and strive for the highest standards of excellence. No matter where you are located, you’ll join a dedicated, diverse community that will help you discover your fullest potential. DESCRIPTION EPAM Big Data Practice is looking for an Azure Big Data Engineers . As an Azure Data Engineer, you will be responsible for designing and implementing the management, monitoring, security and privacy of data using the full stack of Azure data services to satisfy the business needs. You are curious, persistent, logical and clever – a true techie at heart. You enjoy living by the code of your craft and developing elegant solutions for complex problems. If this sounds like you, keep reading to learn more about this exciting role Come and join EPAM where Engineering is in our DNA. RESPONSIBILITIES Implement non -relational data stores: Implement a solution that uses Cosmos DB, Data Lake Storage Gen2 Implement data distribution and partitions Implement a consistency model in Cosmos DB Provision a non-relational data store Provide access to data to meet security requirements Implement for high availability, disaster recovery, and global distribution Implement relational data stores: Provide access to data to meet security requirements Implement for high availability, disaster recovery, and global distribution Implement data distribution and partitions for Azure Synapse Analytics Manage data security: Implement data masking Encrypt data at rest and in motion Develop batch processing solutions: Develop batch processing solutions by using Data Factory and Azure Databricks Ingest data by using Stream Sets and/or ADF Implement the integration runtime for Data Factory Create linked services and datasets Create pipelines and activities Create and schedule triggers Implement Azure Databricks clusters, notebooks, jobs, and autoscaling Ingest data into Azure Databricks Develop streaming solutions: Configure input and output Select the appropriate windowing functions Implement event processing by using Stream Analytics Ingest and query streaming data with Azure Data Explorer Monitor data processing: Monitor Data Factory pipelines Monitor Stream Analytics Monitor Stream Analytics Optimize Azure data solutions: Troubleshoot data partitioning bottlenecks Optimize Data Lake Storage Optimize Stream Analytics Optimize Azure Synapse Analytics Optimize SQL Database Manage the data lifecycle REQUIREMENTS 5-10 years in IT 2-3 years in Azure Experience in Agile or PMI methodology managed projects Experience in enterprise applications, solutions and data infrastructures Experience in designing of data management solutions Experience in designing of robust CI/CD solutions Python/PySpark -highly desired ADLS (gen 1, gen 2 - preview mode) ADF v2 - two instances per project: ingestion and transformation Databricks SQLDB Synapse Analytics (former SQLDW) AAS Embedded Power BI Java/Scala – Spark Git, GitLab, Artifactory ADO Microsoft Certified: Azure Data Engineer Associate SCRUM Developer Azure DevOps WHAT WE OFFER We offer a range of discretionary benefits from time to time, including: Group personal pension plan, life assurance and income protection Private medical insurance, private dental care and critical illness cover Cycle scheme and season ticket loan Employee assistance program Gym discount, Friday lunch, on-site massage and social events 1 day off for your wedding and baby basket Tech purchase scheme Unlimited access to LinkedIn learning solutions Some of these benefits may be available only after you have passed your probationary period This job was originally posted as www.totaljobs.com/job/89906856

People also viewed