Senior Data Engineer - Mumbai, India - RapidBrains

Description

Job Description:

Responsibilities:

• Design, develop, and maintain data pipelines and infrastructure.
• Collaborate with the data engineering team to devise and deploy innovative data-driven solutions.
• Liaise with stakeholders to gather requirements and translate them into technical specifications.
• Troubleshoot and rectify issues within data pipelines and infrastructure.
• Monitor and enhance the performance of data pipelines and infrastructure.
• Stay abreast of the latest advancements in data engineering technologies and trends.

Qualifications:

• Bachelor's degree in computer science, information technology, or a related field.
• 3+ years of experience in data engineering.
• Proficiency in Apache Spark.
• Experience with Databricks data platform is mandatory.
• Familiarity with Azure Data toolset.
• Knowledge of big data technologies like Hadoop and Kafka.
• Expertise in data warehousing, data lakes, data modeling, and data quality.
• Proficiency in SQL and NoSQL databases.
• Experience with cloud computing platforms such as AWS, Azure, and GCP.
• Strong problem-solving and analytical skills.
• Excellent communication and interpersonal abilities.

Desired Qualifications:

• Master's degree in computer science, information technology, or a related field.
• Experience with machine learning and artificial intelligence is advantageous.
• Proficiency in distributed systems.
• Familiarity with open-source software development.
• Experience with Agile development methodologies.

Skills & Requirements:

Databricks with Spark, Azure, data engineering, Apache Spark, Azure Data toolset, big data technologies such as Hadoop and Kafka, data warehousing and data lakes, data modeling and data quality, SQL and NoSQL, cloud computing platforms, problem-solving and analytical skills, communication and interpersonal skills.