Data Engineer - India - Zorba Consulting India

Description

As a Data Engineer specializing in Databricks and Apache Spark, you will be responsible for designing, building, and maintaining scalable data pipelines and analytics solutions. The ideal candidate will have a strong background in data engineering, with hands-on experience in Databricks and Spark. You will collaborate closely with cross-functional teams to deliver high-quality data solutions that meet the needs of our clients.

Responsibilities:

• Design, develop, and optimize data pipelines using Databricks and Apache Spark.
• Implement data processing and transformation logic to support analytics and reporting requirements.
• Work closely with data scientists and analysts to understand data requirements and deliver actionable insights.
• Perform data modeling and schema design to ensure data quality, consistency, and performance.
• Collaborate with infrastructure and DevOps teams to ensure reliability and scalability of data platforms.
• Conduct performance tuning and optimization of Spark jobs to maximize efficiency and throughput.
• Stay updated on emerging technologies and best practices in data engineering and analytics.

Requirements:

• Bachelor's degree in Computer Science, Engineering, or related field.
• 5+ years of experience in data engineering, with a focus on Databricks and Apache Spark.
• Strong proficiency in SQL and Scala/Python programming languages.
• Hands-on experience with building and optimizing data pipelines using Databricks platform.
• Solid understanding of distributed computing principles and techniques.
• Experience with cloud platforms such as AWS, Azure, or GCP.
• Excellent problem-solving and analytical skills.
• Strong communication and collaboration skills.
• Ability to work effectively in a fast-paced, dynamic environment.