prasanna kumar ganesan

I am a results-driven Data Engineer with over 9 years of experience, including 4 years specializing in Big Data and cloud-based data engineering and 5 years in database development. My expertise lies in leveraging Azure Databricks, Azure Data Factory, and Apache Spark to design and implement scalable, secure, and high-performance data pipelines.

Designed and maintained end-to-end ETL pipelines using

ADF to orchestrate data ingestion, transformation, and loading

across Azure SQL Database, Data Lake Storage.

➔ Optimized PySpark ETL jobs processing large datasets by

configuring Spark parameters, implementing effective data

partitioning, and using caching strategies, resulting in a 50%

improvement in job performance.

➔ Established multi-layered data lakehouse architecture

using Azure Databricks and Delta Lake, streamlining ETL

workflows and improving data reusability across teams.

➔ Implemented data partitioning and bucketing in PySpark for

optimal query performance on large datasets, improving

job execution speed and minimizing shuffle operations.

2010 – 2014 B.E(ECE)