Azure Data Engineer - Chennai, India - SKYLARK HR SOLUTIONS Pvt Ltd

Description

JOB DESCRIPTION:

Lead the design, development, and maintenance of Prochant Azure-based database

systems, ensuring optimal performance, scalability, and reliability.

Expert in configuring Azure tool of Azure datalake, Synapse and dbt.

Expert in writing dynamic query on Parquet file using Python.

Expert in creating partition and materialized view.

Collaborate with cross-functional teams, including data scientists, software engineers, and

analysts, to understand data requirements and implement efficient database solutions.

Develop data pipelines and ETL processes to ingest, transform, and load data from various

sources into our Azure environment using technologies such as DBT, Airflow, and Blob

Storage.

Optimize SQL queries and data models to ensure high performance and efficient data

retrieval.

Implement and manage Azure Synapse Analytics to enable advanced analytics and reporting

capabilities.

Utilize Parquet file format and other efficient data storage techniques to optimize data

storage and processing.

Design and implement data warehouse solutions using star schema, facts dimension

modeling, and other data warehousing best practices.

Monitor database systems, identify and resolve performance issues, and implement

necessary improvements to ensure data integrity and availability.

Collaborate with the AI/MI Data Engineering team to integrate machine learning and

artificial intelligence capabilities into our data infrastructure.

Utilize prior experience in SSIS technologies for data integration, data transformation, and

workflow automation.

Stay updated with the latest trends and advancements in database technologies, Azure

Cloud Platform, and data engineering practices.

Skills:

Strong experience in designing, implementing, and managing database systems, particularly

within the Azure Cloud Platform.

Expertise in Azure services such as DBT, Airflow, Blob Storage, Serverless SQL Pool, and

Synapse Analytics.

Proficiency in optimizing SQL queries, data models, and database performance.

Solid understanding of ETL processes, data warehousing, and data integration techniques.

Familiarity with Parquet file format and efficient data storage techniques.

Background experience in data warehouse designs, star schema, and facts dimension

modeling.

Prior experience in SSIS technologies for data integration, transformation, and workflow

automation.

Great to have experience in AI/MI Data Engineering, including integrating machine learning

and AI capabilities into data infrastructure.

Strong problem-solving skills and ability to work independently and collaboratively in a team

environment.

Excellent communication and interpersonal skills to effectively collaborate with cross-

functional teams.

Proven ability to manage multiple priorities and deliver high-quality results within

deadlines.