Aws De- Airflow - Chennai, Tamil Nadu, India - Willware Technologies
Description
Work mode:
One week of onboarding at Chennai office and later Remote
- Design, develop, and deploy complex data pipelines using Apache Airflow on AWS infrastructure.
- Architect, optimize, and automate data workflows to ensure reliability, scalability, and performance.
- Collaborate with data scientists, analysts, and other stakeholders to understand requirements and translate them into technical solutions.
- Implement best practices for data ingestion, processing, transformation, and storage.
- Monitor pipeline performance, troubleshoot issues, and implement enhancements to improve efficiency and reliability.
- Ensure data quality and integrity throughout the data lifecycle by implementing robust validation and error handling mechanisms.
- Stay current with emerging technologies and industry trends in data engineering and contribute to continuous improvement initiatives.
- Mentor junior team members and provide technical guidance and support as needed.
Qualification:
- 57 years of professional experience in data engineering, with a focus on building and managing data pipelines.
- Extensive handson experience with Apache Airflow, including DAG design, scheduling, and orchestration.
- Strong proficiency in AWS services such as S3, EC2, EMR, Glue, Lambda, and Redshift.
- Solid understanding of data warehousing concepts, ETL processes, and data modeling techniques.
- Proficiency in programming languages such as Python, SQL, and Shell scripting.
Job Type:
Temporary
Contract length: 3-6 months
Benefits:
- Work from home
Schedule:
- Monday to Friday
Experience:
- total work: 5 years (required)
Spark Airflow: 5 years (required)
Work Location:
In person
Speak with the employer
More jobs from Willware Technologies
-
Ui/ux Designer
Bengaluru, India - 2 weeks ago
-
SAP Abab with SAP Mm/ SAP Sd/ SAP Fico
Bengaluru, India - 2 weeks ago
-
SAP Mes and Mii
Bengaluru, India - 3 weeks ago
-
Freelancer UI Next Js SEO Expert
Bengaluru, India - 3 weeks ago
-
Contractual devops with Machine Learning onsite
Bengaluru, India - 2 days ago
-
Contractual Role
Bengaluru, Karnataka, India - 3 weeks ago