- Strong expertise in PySpark and Apache Spark for batch and real-time data processing.
- Experience in designing and implementing ETL pipelines, including data ingestion, transformation, and validation.
- Proficiency in Python for scripting, automation, and building reusable components.
- Hands-on experience with scheduling tools like Airflow or Control-M to orchestrate workflows.
- Familiarity with AWS ecosystem, especially S3 and related file system operations.
- Strong understanding of Unix/Linux environments and Shell scripting.
- Experience with Hadoop, Hive, and platforms like Cloudera or Hortonworks.
- Ability to handle CDC (Change Data Capture) operations on large datasets.
- Experience in performance tuning, optimizing Spark jobs, and troubleshooting.
- Strong knowledge of data modeling, data validation, and writing unit test cases.
- Exposure to real-time and batch integration with downstream/upstream systems.
- Working knowledge of Jupyter Notebook, Zeppelin, or PyCharm for development and debugging.
- Understanding of Agile methodologies, with experience in CI/CD tools (e.g., Jenkins, Git).
- Experience in building or integrating APIs for data provisioning.
- Exposure to ETL or reporting tools such as Informatica, Tableau, Jasper, or QlikView.
- Familiarity with AI/ML model development using PySpark in cloud environments
- Skills: ci/cd,zeppelin,pycharm,pyspark,etl tools,control-m,unit test cases,tableau,performance tuning,jenkins,qlikview,informatica,jupyter notebook,api integration,unix/linux,git,aws s3,hive,cloudera,jasper,airflow,cdc,pyspark, apache spark, python, aws s3, airflow/control-m, sql, unix/linux, hive, hadoop, data modeling, and performance tuning,agile methodologies,aws,s3,data modeling,data validation,ai/ml model development,batch integration,apache spark,python,etl pipelines,shell scripting,hortonworks,real-time integration,hadoop.
- Mandatory Key Skills - Apache Spark,Python,ETL,Unix,Linux,data engineering,Agile methodologies,CI/CD,data modeling,data validation,PySpark.
-
We're currently looking for a high caliber professional to join our team as Pyspark-Developer-Hybrid based in Pune, India. · Being part of our team means that we'll provide you with the resources to meet your unique needs, · empower you to make healthy decision and manage your fi ...
Pune Full time3 weeks ago
-
Developing and optimizing data pipelines for large-scale data processing. · ...
Pune Full time2 weeks ago
-
A PySpark developer is required to develop and optimize data pipelines for large-scale data processing. Experience with Python and PySpark is necessary. A bachelor's degree or equivalent experience is required. ...
Pune Full time2 weeks ago
-
Responsibilities · Develop and maintain scalable data pipelines using Python and PySpark. · Collaborate with data engineers and data scientists to understand and fulfill data processing needs. · Optimize and troubleshoot existing PySpark applications for performance improvements. ...
Pune ₹800,000 - ₹2,200,000 (INR) per year Full time13 hours ago
-
+Job summary · Citi está creando tecnología para el futuro del banco. Se busca Pyspark-Developer - Hybrid. · +ResponsibilitiesUtilizar conocimientos de procedimientos y conceptos de desarrollo de aplicaciones. · Consultar con usuarios y clientes sobre problemas. · ...
Pune, Maharashtra3 weeks ago
-
Choosing Capgemini means choosing a company where you will be empowered to shape your career in the way you'd like, where you'll be supported and inspired by a collaborative community of colleagues around the world, and where you'll be able to reimagine what's possible. Join us a ...
Pune, Maharashtra ₹800,000 - ₹2,200,000 (INR) per year20 hours ago
-
Pyspark developer needed for big data environment using Hadoop clusters or cloud. · ...
Pune3 weeks ago
-
At Citi we're not just building technology, we're building the future of banking. · ...
Pune Full time2 weeks ago
-
The Applications Development Intermediate Programmer Analyst is an intermediate level position responsible for participation in the establishment and implementation of new or revised application systems and programs in coordination with the Technology team. · ...
Pune Full time1 month ago
-
· Coding and software engineering: write clean efficient well-documented code primarily in Python PySpark and Java often utilizing frameworks like Spring Boot. · Collaboration and communication work with cross-functional teams including senior developers data engineers analysts ...
Pune Full time1 month ago
-
We are seeking a highly skilled Sr. Developer with 7 to 10 years of experience to join our team. · The ideal candidate will have expertise in Databricks Unity Catalog Azure DevOps Python Databricks SQL Databricks Workflows and PySpark. · Develop and implement robust data solution ...
Pune1 month ago
-
TCS invited applicants for Python Pyspark Developer position. Key responsibilities include designing and developing robust data pipelines using PySpark on AWS. · ...
Pune, Maharashtra1 month ago
-
The Applications Development Intermediate Programmer Analyst is an intermediate level position responsible for participation in the establishment and implementation of new or revised application systems and programs in coordination with the Technology team. The overall objective ...
Pune, Maharashtra20 hours ago
-
We are seeking a highly skilled Sr. Developer with 7 to 10 years of experience to join our team. · ...
Pune, Maharashtra1 month ago
-
About Client :- · Our client is a French multinational information technology (IT) services and consulting company, headquartered in Paris, France. Founded in 1967, It has been a leader in business transformation for over 50 years, leveraging technology to address a wide range of ...
Pune, India Full time4 days ago
-
NielsenIQ is seeking a Python Engineer with hands-on Python & Azure experience. · ...
Pune, MH, India1 month ago
-
The Applications Development Intermediate Programmer Analyst is an intermediate level position responsible for participation in the establishment and implementation of new or revised application systems and programs in coordination with the Technology team. The overall objective ...
Pune Full time7 hours ago
-
We are looking for a skilled and experienced PySpark Data Engineer to join our growing data engineering team. · Design, develop, and maintain large-scale ETL pipelines using PySpark and AWS Glue. · Orchestrate and schedule data workflows using Apache Airflow. · ...
Pune, Maharashtra1 month ago
-
A Application Developer will be responsible for designing and configuring applications to meet business process and application requirements using Ab Initio. · Design build and configure applications using Ab Initio to meet business process and application requirements. · ...
Pune3 weeks ago
-
We are looking for Python Developer with 3 plus years experience in data processing and backend development. · ...
Pune1 month ago
PySpark Developer - Pune - confidential
Description
Key Skills & Responsibilities
Preferred Skills
-
Pyspark Developer
Full time Only for registered members Pune
-
PySpark Developer
Full time Only for registered members Pune
-
PySpark Developer
Full time Only for registered members Pune
-
Pyspark Developer
Full time Tech Mahindra Limited- Pune
-
Pyspark Developer
Only for registered members Pune, Maharashtra
-
Pyspark Developer
Only for registered members Pune, Maharashtra
-
Pyspark Developer
Only for registered members Pune
-
PySpark Developer
Full time Only for registered members Pune
-
Bigdata Developer with PySpark
Full time Only for registered members Pune
-
Bigdata Developer with PySpark
Full time Only for registered members Pune
-
Databricks + Pyspark Developer
Only for registered members Pune
-
Python Pyspark Developer
Only for registered members Pune, Maharashtra
-
Bigdata Developer with PySpark
Only for registered members Pune, Maharashtra
-
Databricks + Pyspark Developer
Only for registered members Pune, Maharashtra
-
Snowflake+Pyspark developer
Full time confidential- Pune, India
-
Python PySpark Developer
Only for registered members Pune, MH, India
-
Big Data Pyspark Developer
Full time Only for registered members Pune
-
Pyspark AWS Glue Developer
Only for registered members Pune, Maharashtra
-
Application Developer( Pyspark,Unix, Ab Initio)
Only for registered members Pune
-
Python Developer Data Processing ETL Pandas PySpark
Only for registered members Pune