beBee background
Professionals
>
Pune
Yash Chilwar

Yash Chilwar

Data Engineer

Technology / Internet

Pune, Pune

Social

About Yash Chilwar:

Data Engineer at Whiteklay Technologies Pvt Ltd.

Experience

Software Development Engineer - 1, Whiteklay Technologies Pvt Ltd – Pune July 2022 - Present
• Working on Star Health and Allied Insurance Co Ltd Project (Big Data Engineering Project)
• Developed robust data ingestion pipelines using Azure Synapse Analytics Pipelines and Copy Data Activity to
extract and load batch data (Full Refresh, SCD0, SCD1) from diverse sources (Oracle, PostgreSQL, MySQL, SQL
Server, MongoDB, SFTP) into Azure Data Lake Storage Gen2 (ADLS Gen2).
• Implemented real-time data streaming using Azure Event Hub to capture and ingest Change Data Capture
(CDC) data into Azure Data Lake Storage Gen2 (ADLS Gen2)
• Developed comprehensive efficient data transformation ,reports and aggregates using PySpark/Scala Spark to
meet specific client requirements and provide actionable insights
• Proactively resolved production pipeline failures, achieving a significant reduction from 76% to 98%,
minimizing downtime and ensuring continuous data availability.
• Engineered and implemented Delta Lake utilities for Repartitioning and Vacuuming, resulting in a 30% boost in
query performance.
• Implemented Rollup/Reconciliation processes, enhancing data accuracy by 60% through the identification and
resolution of source system discrepancies (timezone, late data, primary key conflicts).
• Executed the migration of Synapse Spark notebooks to Databricks Notebooks, delivering a 14% reduction in
operational costs.
Intern, Eaton Technologies Pvt Ltd – Pune Jan 2022 - Jun 2022
• Developed Desktop Web Applications for internal company activities and management using Microsoft Power
Apps. Developed workflows through Microsoft Power Automate for seamless task automation

Education

Vishwakarma Institute of Information Technology
B. Tech, Computer Engineering
CGPA: 9.63

Professionals in the same Technology / Internet sector as Yash Chilwar

Professionals from different sectors near Pune, Pune

Other users who are called Yash

Jobs near Pune, Pune

  • Data Engineer

    4 days ago

    Sulzer Pune, Maharashtra

    We are seeking a skilled and motivated Data Engineer to join our BI & Analytics team at our Pune office. In this role, you will lead the design and implementation of scalable, reusable, and optimized end-to-end data pipelines. · ...

  • Data Engineer

    2 weeks ago

    Syngenta Group Pune, Maharashtra

    We are looking for a Data Engineer to develop and maintain robust data pipelines and infrastructure. · Develop and maintain robust data pipelines and infrastructure. · Enable accurate, real-time, and predictive insights by integrating and transforming data from diverse sources. · ...

  • Data Engineer

    1 month ago

    Tlc Engineering Solutions Ltd Pune City

    This is a full-time Data Engineer role at Toyota Material Handling India Pvt. Ltd. · Designing, building, and maintaining data pipelines and ETL processes to support business operations · Developing and implementing efficient data models, enabling data warehousing solutions · ...

See jobs in Pune, Pune