Lead Data Engineer - Gurugram, India - True Tech Professionals

Description

Lead DataEngineerLocation:
Gurgaon(Hybrid)

Key Skills:
Python Pyspark AWSServices (Glue Athena Redshift)

SQLKeyResponsibilities:
Conceive and implementefficient data models to ensure data accuracy and optimizeperformance

ETLDevelopment:
Developmaintain and enhance ETL processes to efficiently extract transformand load data from diverse sources into our datawarehouse

SQLMastery:
Craftintricate SQL queries for data extraction manipulation and analysisas per project requirements

PythonDevelopment:
Createand maintain Python scripts and applications to support dataprocessing and automation

AWSProficiency:
Utilizeextensive knowledge of AWS services including S3 Redshift Glue EMRand Athena to construct and manage data pipelines andinfrastructure.
Experiencewith tools like Terraform or CloudFormation to automate theprovisioning and management of AWS resources isadvantageous

Big DataProcessing:
Familiaritywith PySpark for largescale data processing and analysis is adesirable skill

Source CodeManagement:
EmployGit and GitHub for version control and seamless collaboration ondata engineering projects

PerformanceOptimization:
Identifyand implement optimizations for data processing pipelines toenhance efficiency and reduce operationalcosts

DataQuality:
Establishdata quality checks and validation procedures to ensure dataintegrity ismaintained

Collaboration:
Collaborateclosely with data scientists analysts and crossfunctional teams tocomprehend data requirements and deliver highquality datasolutions

Documentation:
Maintaincomprehensive documentation for all data engineering processes andprojects

Qualifications:
6to 8 years of experience in data engineering roles with a strongemphasis on AWS technologies.
Proficiency indata modeling SQL and Python.

Demonstratedexpertise in AWS services particularly in the context of dataprocessing and extracttransformload(ETL).Experience with PySpark and big dataprocessing is considered a valuableasset.

Strong version control skills usingGit and GitHub.
Excellent problemsolving andcommunication skills.
Ability to work bothindependently and collaboratively within a team taking ownership ofprojects and delivering ontime.

python,aws,glue,spark,dataengineering,sql,etl,data processing,data