PySpark Developer - Gurgaon/Gurugram, India - Waytogo Consultants

Waytogo Consultants Gurgaon/Gurugram, India

3 days ago

permanent Technology / Internet

Description

Responsibilities :

Design and develop PySpark applications using Spark DataFrames and SQL to extract, transform, and load (ETL) data.
Read data from various sources such as relational databases, data lakes, and cloud storage using PySpark connectors.
Clean, transform, and enrich data using PySpark functions for data wrangling and manipulation.
Build and maintain robust data pipelines using PySpark for efficient data processing.
Optimize PySpark applications for performance and scalability to handle large datasets effectively.
Collaborate with engineers and data scientists to understand data requirements and develop data-driven solutions.
Write unit tests for PySpark applications to ensure code quality and maintainability.
Document PySpark code and applications clearly and concisely for future reference and knowledge sharing.

Qualifications :

Bachelor's degree in Computer Science, Information Technology, or a related field (or equivalent experience).
2+ years of experience in developing big data applications using PySpark.
Strong proficiency in Python programming language and object-oriented programming concepts.
In-depth understanding of Apache Spark architecture, including Spark DataFrames, Spark SQL, and distributed processing.
Experience working with big data platforms such as Hadoop, YARN, and data lakes (e.g., AWS S3, Azure Data Lake Storage).
Experience with cloud platforms (AWS, Azure, GCP) is a plus.

)

PySpark Developer

1 day ago

Freelance Noida, India

Collance Technologies is captive IT office of US parent group - Care Connectors Medical Group - my · Job Location - Noida, Sec 62, Stellar IT Park, Work from Office · Must have requirements - Pyspark · 1. Hands on experience in core and advance python programming · 2. Hands on w ...
QA (Aws Testing) Engg

1 week ago

Zensar Technologies Gurugram, Haryana, India

**JOB DESCRIPTION**: · - Resource will be working with your cloud development team that works on running EMR batches, Lambda functions, PySpark, Python etc Selenium with Java · **Primary Location**: India-Haryana-Gurgaon · **Job Posting**: Apr 12, 2024 · **Experience Required (In ...
Data Scientist

4 days ago

Elvira Technologies Gurugram, Haryana, India

**Skills & Knowledge-** · - **Python, Spark, Azure Databricks /Data factory/DevOps, SQL**: · - Understanding of developing pipeline in Azure data factory · - Ability to write code in **spark **if required · - should have clarity of thought and strong communication skills (verbal ...
Data Eng

3 days ago

Boston Scientific Corporation Gurgaon, India

**Data Eng - P2**: · - Remote Eligible: Hybrid- Onsite Location(s): Gurgaon, HR, IN**Additional Locations**: India-Haryana, Gurgaon · **Diversity - Innovation - Caring - Global Collaboration - Winning Spirit - High Performance** · At Boston Scientific, we'll give you the opportun ...
Economics Analyst-3

1 week ago

Mastercard Gurugram, Haryana, India

Our Purpose · We work to connect and power an inclusive, digital economy that benefits everyone, everywhere by making transactions safe, simple, smart and accessible. Using secure data and networks, partnerships and passion, our innovations and solutions help individuals, financi ...
Economics Analyst-2

4 days ago

Mastercard Gurugram, Haryana, India

Our Purpose · We work to connect and power an inclusive, digital economy that benefits everyone, everywhere by making transactions safe, simple, smart and accessible. Using secure data and networks, partnerships and passion, our innovations and solutions help individuals, financi ...
Data Scientist

4 days ago

Adidas Gurgaon, India

The adidas Digital Data Science team is looking for talents. We are passionate about data and algorithms lifting insights and actions hidden in large data sets. We work on a variety of use cases, from forecasting to product recommendations, from pricing to allocation. We help our ...
Data Scientist

2 days ago

Adidas Gurugram, Haryana, India

Data Scientist - M4 · Gurgaon- The adidas Digital Data Science team is looking for talents. We are passionate about data and algorithms lifting insights and actions hidden in large data sets. We work on a variety of use cases, from forecasting to product recommendations, from pri ...
Associate- Data Scientist

3 days ago

Bain & Company Gurgaon, India

**ABOUT US**: · Bain & Company is a global consultancy that helps the world's most ambitious change makers define the future. Across 61 offices in 39 countries, we work alongside our clients as one team with a shared ambition to achieve extraordinary results, outperform the compe ...
Data Scientist

5 days ago

United Airlines Gurugram, Haryana, India

**Description**: · United's Digital Technology team designs, develops, and maintains massively scaling technology solutions brought to life with innovative architectures, data analytics, and digital solutions. · **Our Values**: At United Airlines, we believe that inclusion propel ...
Scientist - Data Platform Optimizer

1 week ago

AXA Gurgaon, India

AXA XL recognizes digital, data, and information assets are critical for the business, both in terms of managing risk and enabling new business opportunities. Data Science and Applied AI assets should not only be high quality but also drive a sustained enterprising advantage and ...
Data Scientist

1 week ago

BAXTER Gurgaon, India

**This is where you save and sustain lives** · At Baxter, we are deeply connected by our mission. No matter your role at Baxter, your work makes a positive impact on people around the world. You'll feel a sense of purpose throughout the organization, as we know our work improves ...
Scientist - Data Platform Optimizer

1 week ago

XL Catlin Gurgaon, India

**Flexible Work Eligible**: None · **DISCOVER your opportunity** · AXA XL recognizes digital, data, and information assets are critical for the business, both in terms of managing risk and enabling new business opportunities. Data Science and Applied AI assets should not only be ...
Associate - Projects

4 days ago

Cognizant Technology Solutions Gurgaon, Gurugram, Haryana, India

**Technical Lead** · **Qualification**: · Bachelors in science, engineering or equivalent · ** Responsibility**: · **Project Planning and Setup**: · - Understand the project scope, identify activities/ tasks, task level estimates, schedule, dependencies, risks and provide inputs ...
Associate - Projects

4 days ago

Cognizant Technology Solutions Gurgaon, Gurugram, Haryana, India

**Technical Lead** · **Qualification**: · Bachelors in science, engineering or equivalent · ** Responsibility**: · **Requirement Understanding and Analysis**: · - Understand the functional/ nonfunctional requirements. · - Participate in client calls and prepare the clarification ...
Associate Data Scientist

1 week ago

Gartner Gurgaon, India

**Description**: · **About the team**: · **This is a unique opportunity for a creative and charismatic individual to lead a team of data scientists of a data science research and product development team for Gartner Digital Products and Innovations Group, a fast-growing data scie ...
Data Scientist

2 days ago

S&P Global Gurugram, Haryana, India

**About the Role**: · **Grade Level (for internal use)**: 10 · **Position title**: Data Scientist · **Years of Experience**: 4-8 years · **Location**: Gurgaon · **Department**: Carfax - Business Intelligence · **Shift Timings**: 2:00PM to 11:00PM IST · Join our Winning Team as Da ...
Product Owner Data Foundations

1 day ago

Adidas Gurgaon, India

Product Owner Data Foundations- Area: Digital · Job family: Product Ownership · GSMS Grade: M4 · Number of Personnel Managed: n/a · Cost Center/Budget and/or Revenue Responsibility: tbd- Department: Global Sales - Digital - Digital Growth · Direct Reporting Line: Senior Product O ...
Python +pyspark

1 day ago

Nityo Infotech Delhi, India

2 hours ago · **Job Code**: · JD-19623 · **JOB DESCRIPTION**: · Mandatory Skills - Python, Pyspark, DataBricks, SQL Primary Skills: - Hands-on Python, PySpark, Databricks, SQL, AWS (S3, Lambda, EC2, RDS), and CI-CD tools. Job description: - 6 or more years of experience developin ...
Pyspark

3 days ago

Iris Software Noida, India

**Posted On**:24 Jan 2024 · **Location**: Noida, UP, India · **Company**:Iris Software · **Why Join Us?** · Are you inspired to grow your career at one of **India's Top 25 Best Workplaces in IT industry? **Do you want to do the best work of your life at one of the **fastest growi ...