Ashutosh Yadav

I am an experienced Data Engineer with a passion for leveraging cloud-based technologies to build data-driven solutions. I have a strong background in designing, implementing, and maintaining data pipelines using Azure technologies. 

Lentra AI (April 2023- Present):

• Led the migration of legacy data infrastructure to Apache Pinot, enabling real-time data analytics and efficient querying capabilities.
• Utilised Bitbucket for version control, collaborating with the development team to manage code repositories and streamline the deployment process.
• Collaborated with cross-functional teams, including data scientists, analysts, and product managers, to understand the business requirements and translate them into effective data models.
• Designed and implemented schema structures in Apache Pinot, ensuring optimal data storage, query performance, and scalability to meet analytical demands.
• Conducted knowledge sharing sessions and workshops to educate team members about Apache Pinot's capabilities and how to effectively leverage its features.

Celebal Technologies (Sept 2021- March 2023)

• Developed Spark applications using Pyspark and Spark-SQL for data extraction, transformation, and aggregation from multiple file formats for analyzing & transforming the data to uncover insights into customer usage patterns.
• Extract Transform and Load data from Sources Systems to Azure Data Storage services using a combination of Azure Data Factory, and, Spark SQL. Data Ingestion to one or more Azure Services - (Azure Data Lake, Azure Storage) and processing the data in Azure Databricks.
• Participated in data quality checks and monitoring to ensure data integrity and accuracy.
• Created Pipelines in ADF using Linked Services/Datasets/ Pipeline/ to Extract, Transform and load data from different sources like Blob Storage, Azure SQL Data Warehouse, write- back tool, and backward.
• Automated ELT processes, via email to destination using Azure Logic Apps and Data factory and translated the data into actionable reports in Power BI.
• Generated & maintained scalable data pipelines and built new API integrations to support data volume & complexity. Data Transformations were done on Azure Databricks.
• Collaborated with the analytics team to enhance business intelligence tools. Implemented processes to monitor data quality and ensured accurate production data by 100%

Masters in Computer Applications

• Vellore Institute of Technology       (2020 - 2022)

Bachelors of Science in Information Technology

• Mumbai University (2016 - 2019)