Azure Databricks, PySpark, SQL - Bengaluru, India - Infogain

Description

JOB DESCRIPTION

Position: Senior / Lead Engineer

Overall IT experience: 6 to 12 years

Primary Skills: Azure Databricks, PySpark, SQL

Secondary Skills: ADF/Azure Datafactory

Work Location: Bengaluru, Pune, Mumbai, Noida, Gurugram

Mode of work: Hybrid at Infogain office locations mentioned above

Notice Period: 0 to 30 Days

Essential experience:

• Overall 6 to 12 years of experience Cloud data engineering, out of whichminimum of 3 to 5 years experience in Azure Databricks, PySpark.
• in SQL and must worked on complex joins, Subqueries, functions, procedure. and able to perform SQL tunning and query optimization without support. Ideally 5 to 6 years of solid experience in SQL queries
• develop, and maintain ETL pipelines using Databricks, PySpark and ADF to extract, transform, and load data from various sources.
• in implementing large & complex hosted solution using Azure (ADF, Azure Data Lake Storage and databricks.
• and fine-tune existing ETL workflows for performance and scalability.
• have experience on working with large data set
• working with batch and streaming data processing (Good to have).
• to develop reliable, secure, compliant data processing systems.
• closely with cross-functional teams to support data analytics, reporting, and business intelligence initiatives.
• should be self-driven and work independently without support.