- Design, develop, and maintain data pipelines using Azure Functions and Snowpark.
- Utilize SQLAlchemy to interact with relational databases and manage data transformations.
- Implement data validation and error handling using Pydantic data models.
- Automate data pipelines for efficient data processing and delivery.
- Integrate unit and integration testing into data pipelines using pytest.
- Develop and implement a comprehensive testing strategy for data pipelines.
- Monitor data pipelines for errors and ensure data Work closely with the data science team to understand their needs and translate them into technical solutions.
- Document data pipelines and processes for clear communication and maintainability.
- Participate in code reviews and knowledgesharing sessions.
- Strong expertise in Python programming language (including libraries like pandas, NumPy).
- Experience with cloud platforms like Microsoft Azure (Azure Functions a plus).
- Experience with data warehousing and data lake architectures (Snowpark a plus).
- Proficiency in SQL for data querying and manipulation (Selects, Joins, Filters, CTEs).
- Familiarity with data pipeline design and development best practices.
- Excellent communication and collaboration skills.
-
Data Engine
3 days ago
Decathlon India Bengaluru, India**Our Purpose**: *"Move people through the wonders of Sport"*About Decathlon Sports India: · Founded in 1976 in France, we have set about making the pleasure and benefit of sports available for everyone across the globe. We do that by providing world-class products both for enthu ...
-
Data Enginer
2 days ago
NatWest Group Bengaluru, Karnataka, IndiaOur people work differently depending on their jobs and needs. From hybrid working to flexible hours, we have plenty of options that help our people to thrive. · This role is based in India and as such all normal working days must be carried out in India. · Join us as a Data Engi ...
-
Data Engine
3 days ago
Decathlon Bengaluru, IndiaPermanent contract · - Full-time · - Mid-Senior level · - BTEC Higher National Diploma, Diploma of Higher Education · - Data engineer · **Mission**: · **Our Purpose**: · **_"Move people through the wonders of Sport"_** · **About Decathlon Sports India**: · Founded in 1976 in Fran ...
-
Data Engineer-1
4 days ago
Commonwealth Bank Bengaluru, India**Your business**: · The Commonwealth Bank is the leading financial institution in Australia and provides integrated financial services. · The role sits within the Bankwest Technology division. Bankwest is a subsidiary of The Commonwealth Bank, and the Bankwest Technology divisio ...
-
Data Engineer 169368 Full-time
1 week ago
HyreFox Consultants Bengaluru, India Full timeBuild and maintain end-to-end data engineering pipelines on batch data. · - Design and implement batch data ingestion pipelines. · - Design and implement batch data transformation pipelines. · - Design and implement batch data storage solutions. · - Implement batch data governanc ...
-
Azure Data Engineer-3-10years-hyderabad
6 days ago
Black & White Business Solutions Bengaluru, India**Job Information**: · Industry · **IT Services** · *** · Province · **Karnataka** · *** · City · **Bangalore North** · *** · Postal Code · **560001** · *** · Country · **India** · Job Title:Azure data Bricks · Years of EXP:3 to 10Years · Location:Hyderabad · NP:30Days/Immediate ...
-
Sr Data Engineer/Data Engineer
1 week ago
Trigent Software - Professional Services Bengaluru, IndiaSenior Data Engineer (5+ Years) / Data Engineer (3+): · Experience working with distributed technology tools for developing Batch and Streaming pipelines using · SQL, Spark, PySpark, · Airflow [3+ years], · Spark with Scala (optional). · Able to write code which is optimized ...
-
Data Engineer
1 day ago
JRD Systems Bengaluru, IndiaRole : Data Engineer - AWS and Oracle Expert · Job Summary: · We are seeking a highly skilled and experienced Data Engineer with expertise in data development in AWS and Oracle. The ideal candidate will have a minimum of 5 years of experience in data engineering, with a strong fo ...
-
Data Engineer
5 days ago
Course5i Bengaluru, IndiaThe Data Engineer will be responsible for designing, building, and maintaining data pipelines and infrastructure that enable efficient and accurate data processing. · The Data Engineer will work closely with data scientists, analysts, and project teams to ensure data is accessibl ...
-
Data Engineer
1 week ago
Sun Life Bengaluru, IndiaKey Responsibilities · This is a position for AWS data engineer typically works on development, implementing, and maintaining data solutions on the Amazon Web Services (AWS) platform. · Working with the Subject Matter Experts and Business Analysts to grasp the best knowledge of e ...
-
Engineer, Data
4 days ago
Giant Eagle GCC , Karnataka, IndiaJob Summary · As a Data Engineer on the Customer Loyalty & Experience team, you will be working on a team to bring customer-centric personalization to life. In this role, you will be empowered to develop data solutions in support of marketing analytics and data science by engagin ...
-
Data Engineer
2 days ago
Recro Bengaluru, IndiaExperience:3+ years · Work location: Bangalore · primary_skills(Must_have) · Programming Skills: Proficiency in Python, Pyspark, and SQL is essential. · Big Data Experience: Hands-on experience with Spark and relational databases such as Postgres is required. · Orchestration To ...
-
Data Engineer
5 days ago
TEKsystems Bengaluru, IndiaExperience – 4 to 10 years · Requirement- · Lead the technical team, help, and support junior developers in project work. · Expert level in Azure Data-bricks with SQL · Expert level in Pyspark programming. · Good understanding in Azure Data Factory (ADF) · Good understanding on D ...
-
Data Engineer
1 week ago
Decathlon Sports India Bengaluru, IndiaAbout Decathlon Sports India: · Founded in 1976 in France, we have set about making the pleasure and benefit of sports available for everyone across the globe. We do that by providing world-class products both for enthusiastic beginners as well as passionate professionals at extr ...
-
Data Engineer
6 days ago
UNO Digital Bank Bengaluru, IndiaOverview: · We are looking for an experienced data engineer to join our team. You will use various methods to transform raw data into useful data systems. Overall, you'll strive for efficiency by aligning data systems with business goals. · To succeed in this data engineering po ...
-
Data Engineer
6 days ago
Joveo Bengaluru, IndiaWe are looking for talented Data Engineers who are passionate about data engineering and have strong experience in building scalable data pipelines. As a Data Engineer at Joveo, you will play a leading role in designing and building datasets that power analytics for teams across ...
-
Data Engineering
1 week ago
Fractal Bengaluru, IndiaWe are seeking a talented Lead Data Engineering Manager to support a Fortune 500 Technology Client. The ideal candidate will have 7-12 years of experience in setting up & implementing digital analytics (Adobe Analytics). leading data modeling and ETL development, APIs that serve ...
-
Data Engineers
1 week ago
Multplyr Bengaluru, IndiaPayroll Company: Multplyr · Role: Data Engineers · Experience: years, across levels · Location: Hybrid / WFO at our client offices in Bengaluru, Mumbai, Pune, Delhi/NCR, Hyderabad and Chennai · Type: Contract / Contract To Hire · Typical Duration: months · About Us · Multplyr is ...
-
Data Engineer
2 days ago
SMARTBORN SOFTWARE CONSULTING PVT. Bengaluru, IndiaRole Description · This is a full-time hybrid role as a Data Engineer at SMARTBORN SOFTWARE CONSULTING PVT.LTD. The Data Engineer will be responsible for day-to-day tasks related to data engineering, including data modeling, Extract Transform Load (ETL) development, data warehous ...
-
Data Engineer
6 days ago
Ascendion Bengaluru, IndiaJob Title: Data Engineer/ Lead · Duration: Full Time · Location: Bengaluru, Hyderabad, Chennai, Pune (Hyrbrid) · NOTE: Looking for Immediate Joiners ( 2-3 weeks notice) · Proficient in Azure Cloud services (or GCP is Ok) for data engineering tasks. · Familiarity with Kubeflow, wi ...
Python Data Engineer - Bengaluru, India - TERRAGIG LLP
Description
We are searching for a highly skilled Data Engineering Specialist to join our growing team.In this role, you will play a critical role in supporting our data science team by building and maintaining efficient data pipelines and infrastructure.
Your expertise in Python and cloud technologies will be instrumental in ensuring the smooth flow of data and enabling our data scientists to focus on delivering impactful Pipeline Development:Testing and Quality Assurance:
Required Skills: