Suhaib Khan

Data Engineer with a strong foundation in scalable ETL pipelines, real-time data streaming, and cloud-based big data architectures. Experienced with Apache NiFi, Kafka, HDFS, Apache Spark, and Flink for building high-throughput data systems. Proficient in SQL and Python for data wrangling, analysis, and pipeline orchestration. Hands-on with AWS services (including S3 and Athena), and capable of integrating batch and streaming workflows using tools like Airflow. Skilled in simulating enterprise-scale data and delivering actionable insights from raw logs, text, and structured data. Seeking to build robust, efficient, and maintainable data solutions in enterprise environments.

I'm currently working as an Azure Data Engineering Intern at Pianalytix, where I’ve been designing and building real-time data pipelines for fintech and logistics use cases. Most of my work involves tools like Apache Spark, Flink, and Kafka to create low-latency systems for things like fraud detection, volatility tracking, and operational analytics.

One of the highlights so far has been improving anomaly detection accuracy by around 40%. I did this by applying stateful stream processing and enriching the data from multiple sources. It's been a hands-on role that’s helped me deepen my understanding of real-time architectures and cloud data platforms.

I completed my B.Tech in Electronics and Communication Engineering from IIITDM Kancheepuram. During my final year, I worked on a project focused on fiber optic sensing, which had a strong IoT component. That project exposed me to a lot of cloud service providers (like Google firebase and AWS), and I ended up building ETL pipelines to process the sensor data. That hands-on experience with IoT, cloud tools, and data engineering workflows is what sparked my interest in the field and ultimately led me to pursue data engineering full-time.