Data Engineer - ahmedabad, India - Talentoj

Talentoj ahmedabad, India

1 day ago

permanent Technology / Internet

Description

Azure Data Engineer :

We are seeking a highly skilled Azure Data Engineer to join the data engineering team and play a crucial role in an optimization initiative.

The current problem is that read speed is compromised and faster responses from delta are required.

The ideal candidate should have extensive experience in Python programming and be proficient in PySpark and Databricks. The selected candidate will be responsible for designing and implementing efficient data pipelines and solutions for data-driven projects in Azure Data Factory and Azure Databricks

Responsibilities :

Design, build, and maintain scalable data pipelines using PySpark and Databricks
Optimize data processing and storage for maximum performance and efficiency
Troubleshoot and debug datarelated issues, and implement solutions to prevent reoccurrence
Collaborate with data scientists, software engineers, and other stakeholders to ensure that data solutions are aligned with business goals

Requirements :

Strong experience in Python programming and PySpark, and SparkSQL
Clear understanding of Spark Data structures, RDD, Dataframe, dataset
Expertise in Databricks and ADLS
Expertise handling data type, from dictionaries, lists, tuples, sets, arrays, pandas dataframes, and spark dataframes
Expertise working with complex data types such as, structs, and JSON strings.
Clear understanding of Spark Broadcast, Repartition, Bloom index filters
Experience with ADLS optimization, partitioning, shuffling and shrinking
Ideal experience with disk caching
Ideal Experience with cost based optimizer
Experience with data modeling, data warehousing, datalake, deltalake and ETL/ELT processes in ADF
Strong analytical and problemsolving skills
Excellent documentation, communication and collaboration skills
Experience of Leading team of engineers and have mentored them

)

Data Engineer

1 week ago

TIGI HR ® Ahmedabad, India

Position- Data Engineer · Experience- 4-6 Years · Budget- 9.00LPA · Location - Ahmedabad · Notice period- Immediate - 30 Days · Roles & Responsibilities: · Designing and building optimized data pipelines using cutting-edge technologies in a cloud environment to drive analytical i ...
Data Engineer

1 day ago

TIGI HR ® Ahmedabad, India

Position- Data Engineer · Experience-3-6 Years · Budget- 10 LPA · Notice period- immediate - 30 Days · Location- Ahmedabad · Responsibilities: · Design and build optimized data pipelines using cutting-edge technologies in a cloud environment (GCP) for analytical insights. · Const ...
Data Engineer

1 day ago

Paradise Placement Consultancy ahmedabad, India

Responsibilities · development of methods and tools for data validation and analysis and in planning and execution of complex projects · Focus on Quality Analytics (e.g., Mean Time between Failures, Mean Time to Crash, Mean Time to Hang, and other quality metrics) · Troublesh ...
Data Engineer

1 day ago

TIGI HR ® Ahmedabad, India

Position- Data EngineerExperience-3-6 YearsBudget- 10 LPANotice period- immediate - 30 DaysLocation- Ahmedabad · Responsibilities:Design and build optimized data pipelines using cutting-edge technologies in a cloud environment (GCP) for analytical insights.Construct cloud infrast ...
Data Engineer

1 day ago

Techno Wise ahmedabad, India

Our US product based client is looking for few Data Engineers for their Analytics department. · Roles: · Understand design/mapping documents and implement into code using ETL tool/technology preferred by company. · Develop code that adhere to development standards and are a ...
Data Engineer

3 hours ago

Techno Wise Ahmedabad, India

Our US product based client is looking for few · Data Engineers · for their Analytics department. · Roles: · Understand design/mapping documents and implement into code using ETL tool/technology preferred by company.Develop code that adhere to development standards and are accura ...
Data Engineer

3 hours ago

Paradise Placement Consultancy Ahmedabad, India

Responsibilities · development of methods and tools for data validation and analysis and in planning and execution of complex projectsFocus on Quality Analytics (e.g., Mean Time between Failures, Mean Time to Crash, Mean Time to Hang, and other quality metrics)Troubleshoot comple ...
Data Engineer Trainer

1 week ago

Helix Tech IT Services Ahmedabad, India

About the Role: · We are seeking a highly skilled and motivated Data Engineer Trainer to join our dynamic training team. This role is ideal for someone with a strong background in data engineering and a passion for teaching and mentorship. The Data Engineer Trainer will be respon ...
Principal Data Engineer

1 day ago

Paradise Placement Consultancy ahmedabad, India

Responsibilities · Guide the Data Engineering team in development of methods and tools for data validation and analysis and in planning and execution of complex projects · Focus on Quality Analytics (e.g., Mean Time between Failures, Mean Time to Crash, Mean Time to Hang, and ...
Senior Data Engineer

1 day ago

Ridgeant Techonologies Ahmedabad, India

Job Description : AWS Data Engineer · Responsibilities : · - Collaborate with stakeholders to understand business requirements and data needs, and translate them into scalable and efficient data engineering solutions using AWS Data Services. · - Design, develop, and maintain data ...
Sr. Data Engineer

1 day ago

Simform Ahmedabad, India

We are looking for Senior Data Engineer who is passionate about managing large amount of data, eager to learn and accept challenges, and who is committed to delivering exceptional results. If you are a team player, with a positive attitude and a desire to make a difference, then ...
Principal Data Engineer

3 hours ago

Paradise Placement Consultancy Ahmedabad, India

Responsibilities · Guide the Data Engineering team in development of methods and tools for data validation and analysis and in planning and execution of complex projectsFocus on Quality Analytics (e.g., Mean Time between Failures, Mean Time to Crash, Mean Time to Hang, and other ...
Sr. Data Engineer

1 day ago

Simform ahmedabad, India

We are looking for a Data Engineer who is passionate about managing large amounts of data, eager to learn and accept challenges, and who is committed to delivering exceptional results. If you are a team player, with a positive attitude and a desire to make a difference, then we w ...
Sr. Data Engineer

1 day ago

Simform Ahmedabad, India

We are looking for a Data Engineer who is passionate about managing large amounts of data, eager to learn and accept challenges, and who is committed to delivering exceptional results. If you are a team player, with a positive attitude and a desire to make a difference, then we w ...
Sr. Data Engineer

21 hours ago

Simform Ahmedabad, India Full time

We are looking for Senior Data Engineer who is passionate about managing large amount of data, eager to learn and accept challenges, and who is committed to delivering exceptional results. If you are a team player, with a positive attitude and a desire to make a difference, then ...
Big Data Engineer

1 day ago

Paradise Placement Consultancy ahmedabad, India

Responsibilities · : · • Responsible for the architecture planning, design, and core code implementation of the company's monitoring system platform, including product technology pre-research, technology architecture selection, core code writing, design document writing, etc. ...
Senior Data Engineer

1 day ago

Cygnet Infotech ahmedabad, India

Experience:- 4 to 8 years · Description: · Build, maintain, and support large-scale collections and processing systems. · Be part of a Scrum team that designs, swarms, and pairs together · Implement and improve deployment, monitoring, testing, operational and foundational t ...
Big Data Engineer

3 hours ago

Paradise Placement Consultancy Ahmedabad, India

Responsibilities · : · • Responsible for the architecture planning, design, and core code implementation of the company's monitoring system platform, including product technology pre-research, technology architecture selection, core code writing, design document writing, etc. · • ...
AWS Data Engineer/Senior Engineer

1 week ago

Arrow Electronics Ahmedabad, India OTHER

Position: · AWS Data Engineer/Senior EngineerJob Description: · Responsibilities: · Collaborate with stakeholders to understand business requirements and data needs, and translate them into scalable and efficient data engineering solutions using AWS Data Services. · Design, deve ...
Data Loss Prevention Engineer

1 day ago

Adani Group Ahmedabad, India

As a Data Loss Prevention Engineer at Adani, you will play a crucial role in safeguarding our organization's sensitive data and ensuring compliance with cybersecurity regulations. The ideal candidate will have 4-7 years of experience in the cybersecurity industry, with expertise ...