Big Data Developer - Hyderabad, India - Talentgigs

Description

Role: Big Data Engineer.

Experience: 7 Years.

Location: Hyderabad(HYBRID).

NP: Immediate - 15 Days

Mandatory skills: GCP - Cloud - Big Data Engineer, Apache Airflow, Apache Spark, Scala.

Job Description:

Key Responsibilities:

● Design, develop, and maintain Big Data pipelines, ensuring data quality, reliability, and scalability.

● Utilize Apache Spark and Apache Airflow to build and manage data processing workflows.

● Lead and mentor team members in the development of data engineering projects.

● Collaborate with product managers and cross-functional teams to understand business requirements and translate them into data solutions.

● Create and maintain system designs for Big Data pipelines, incorporating best practices and scalability considerations.

● Collaborate with the data science and analytics teams to deliver high-quality data for analysis and reporting.

● Continuously evaluate and adopt new technologies and tools to enhance data engineering capabilities.

Qualifications:

● Bachelor's or Master's degree in Computer Science, Engineering, or a related field.

● Proven experience (5+ years) in developing and implementing Big Data pipelines using Apache Spark and Apache Airflow.

● Experience with Google Cloud Platform (GCP).

● Strong programming skills in Scala, Python, and Java.

● Demonstrated leadership skills with the ability to lead and mentor team members.

● Experience in designing scalable Big Data pipeline systems.

● Excellent problem-solving and critical-thinking abilities.

● Strong communication and collaboration skills.

● Experience working in an Agile or Scrum development environment is a plus.