Azure Data Engineer - Pune, India - Persistent Systems

Description

About Position

We are looking for a dynamic Azure Data Engineer with exposure and hands on exp on Azure Stack – Azure Data Factory, Databricks, Pyspark, Scala. Certified Databricks will be an added advantage.

Exp in healthcare & clinical domain is preferred and good to have.

Role: Azure Data Engineer

Location: All PSL Locations

Experience: 5 to 11 years

Job Type: FTE

What You'll Do

• Developing Modern Data Warehouse solutions using Azure Stack (Azure Data Lake, Azure Databricks) and PySpark.
• Hands-on experience in designing and developing scripts for custom ETL processes and automation in Azure data factory, Azure data bricks, Python, Pyspark etc.
• Build and maintain data pipelines to support large scale data management projects
• Solid hands-on working experience in SQL and scripting.
• Experience in any orchestration/workflow tools for scheduling pipelines
• Utilize Python and Spark within Databricks notebooks for data processing and transformations.
• Integrate and orchestrate data from various sources, ensuring data quality and accuracy.
• Build reusable rules for Data Quality and required transformations.
• Optimize data pipelines for performance and scalability.
• Implement data governance and security measures to comply with regulations.
• Maintain documentation of data pipelines and workflows for knowledge sharing.
• Analyse the data quality, data governance, compliance, and other legal requirements on data storage
• Sound knowledge on Data Ingestion and Integration, Data Transformation and Processing, Data Storage and Management, Data Warehousing, Data Modeling and Analysis, Real-time Data Processing, Data Governance and Security

Expertise You'll Bring

• Developing Modern Data Warehouse solutions using Azure Stack (Azure Data Lake, Azure Databricks) and PySpark.
• Hands-on experience in designing and developing scripts for custom ETL processes and automation in Azure data factory, Azure data bricks, Python, Pyspark etc.
• Build and maintain data pipelines to support large scale data management projects
• Solid hands-on working experience in SQL and scripting.
• Experience in any orchestration/workflow tools for scheduling pipelines
• Utilize Python and Spark within Databricks notebooks for data processing and transformations.
• Integrate and orchestrate data from various sources, ensuring data quality and accuracy.
• Build reusable rules for Data Quality and required transformations.
• Optimize data pipelines for performance and scalability.

Benefits

• Competitive salary and benefits package
• Culture focused on talent development with quarterly promotion cycles and company-sponsored higher education and certifications
• Opportunity to work with cutting-edge technologies
• Employee engagement initiatives such as project parties, flexible work hours, and Long Service awards
• Annual health check-ups
• Insurance coverage: group term life, personal accident, and Mediclaim hospitalization for self, spouse, two children, and parents

Our company fosters a values-driven and people-centric work environment that enables our employees to:

• Accelerate growth, both professionally and personally
• Impact the world in powerful, positive ways, using the latest technologies
• Enjoy collaborative innovation, with diversity and work-life wellbeing at the core
• Unlock global opportunities to work and learn with the industry's best

Let's unleash your full potential at Persistent

"Persistent is an Equal Opportunity Employer and prohibits discrimination and harassment of any kind."