DNACRL_Senior Data Engineer

Senior Data Engineer

We are seeking a highly skilled Senior Data Engineer with 5-8 years of total experience, including a minimum of 5 years specifically in the Data Engineering domain. The ideal candidate will have expertise in Scala, Spark, Pyspark, and Azure technologies, particularly Azure Synapse and Data Lake. This role involves designing scalable Azure data architectures for enterprise analytics and ensuring optimal performance and security of data pipelines.

Key Responsibilities

• Design scalable Azure data architectures using ADLS and Azure Synapse for enterprise analytics utilizing the Spark framework.
• Develop and manage data models across bronze, silver, and gold layers to support reporting and analytics, utilizing Delta/Parquet file formats.
• Optimize jobs, monitor pipelines, and enforce governance to ensure performance, reliability, and security.
• Demonstrate a deep understanding of Apache Spark architecture and distributed data processing concepts.
• Utilize Scala and Python for data processing tasks.
• Work with Azure Synapse and utilize KQL to query Log Analytics.
• Manage source code using GIT/Azure Repos and build tools like Maven.
• Lead and mentor engineers, driving best practices and collaborating with stakeholders throughout the project lifecycle.

Requirements

• Strong expertise in Azure Synapse, Pyspark, Spark, Scala, and Azure SQL.
• Excellent problem-solving and communication skills.
• A degree in a relevant field.

Work Arrangements

• Type: Contract
• Location: Bengaluru, Karnataka, India | Hyderabad, Telangana, India (Pan India project)