Rahul Marathe

Data professional with 10+ years of industry experience and 5+ years of relevant hands-on work in Azure-based data engineering. Experienced in supporting ETL pipelines using ADF and Databricks, implementing PySpark/SQL validations, and ensuring high data quality, data reliability, validation across production systems.

Work Experience: Data Engineer / Big Data Developer, Dassault Systemes Solutions Lab, Pune April 2013 to Present Responsibilities-  Designed and enhanced 20+ Azure Data Factory (ADF) pipelines executing daily and scheduled batch loads, ingesting data from SQL databases and enterprise source systems into ADLS Gen2.  Developed reusable PySpark and Python-based transformation and validation logic in Azure Databricks to cleanse, standardize, and prepare curated datasets for analytics and reporting consumption.  Implemented comprehensive data quality and reconciliation checks (schema validation, null/duplicate detection, source to-target counts), improving data accuracy and reliability across production datasets.  Supported incremental and full-load processing workflows handling GB-level datasets (~25 GB/day), contributing to scalable and efficient big data processing pipelines.  Monitored and troubleshot production ETL pipelines, performing reruns and root-cause analysis, resulting in reduced pipeline failures and improved SLA adherence.  Prepared and validated analytics-ready datasets used by reporting and analytics teams (e.g., operational and master data domains), ensuring alignment with business definitions and governance standards.  Collaborated with data analysts, reporting teams, and senior engineers in an Agile environment to analyze data issues, clarify requirements, and deliver reliable data solutions.  Followed established data governance practices, including naming conventions, folder structures, schema consistency, and controlled access across development, UAT, and production environments.  Supported UAT and Production deployments, validating post-release data accuracy and coordinating fixes to resolve data discrepancies before business sign-off.  Gained hands-on exposure to Spark Structured Streaming concepts for near real-time data processing scenarios. Environment: ADF, ADB, ADLS Gen2, PySpark, Python, SQL, Delta Lake, Git, JIRA Design Engineer, Grasp Technologies Pvt. Ltd, Pune August 2012 to March 2013 Responsibilities- • Worked on engineering data models and specifications, gaining early exposure to structured data interpretation and validation. Quality Control Engineer, Walchandnagar Industries Ltd, Baramati August 2011 to July 2012 Responsibilities- • Ensured compliance with standards and accuracy of inspection data, building a strong foundation in quality and validation practices.

Bachelor of Engineering in Production from KIT College of Engineering, Kolhapur