Data Engineer - Hyderabad/Gurgaon/Gurugram, India - TOTLRPO

Description

Qualifications :

a. 11+ years of overall technology experience that includes at least 4+ years of hands-on software development, data engineering, and systems architecture.

b. 4+ years of experience with Data Lake Infrastructure, Data Warehousing, and Data Analytics tools.

c. 4+ years of experience in SQL optimization and performance tuning, and development experience in programming languages like Python, PySpark, Scala etc.).

d. 2+ years in cloud data engineering experience in Azure.

e. Experience with data modeling, data warehousing, and building high-volume ETL/ELT pipelines.

f. Experience building/operating highly available, distributed systems of data extraction, ingestion, and processing of large data sets.

g . Experience with at least one MPP database technology such as Redshift, Synapse or SnowFlake.

I. Experience with running and scaling applications on the cloud infrastructure and containerized services like Kubernetes.

J. Experience with Azure Data Factory, Azure Databricks and Azure Machine learning tools.

Responsibilities :

a. Active contributor to code development in projects and services.

b. Manage and scale data pipelines from internal and external data sources to support new product launches and drive data quality across data products.

c. Build and own the automation and monitoring frameworks that captures metrics and operational KPIs for data pipeline quality and performance.

d. Evolve the architectural capabilities and maturity of the data platform by engaging with enterprise architects and strategic internal and external partners.

e . Support large-scale experimentation done by data scientists.

f. Prototype new approaches and build solutions at scale.
)