- Design and Develop Data Pipelines: Create robust and efficient (infrastructure and cost efficient) data pipelines on GCP to facilitate the seamless flow of data.
- Cross-functional Collaboration: Collaborate with cross-functional teams/stakeholders to understand data requirements and deliver high-quality data solutions in the least amount of time.
- Collaborate with Data Architects: Work closely with client and team data architects to implement data models and pipelines and ensure their integration with GCP services.
- Optimize Data Storage Solutions: Utilize BigQuery, Cloud Storage, and other appropriate GCP services to optimize data storage solutions.
- Troubleshooting: Proactively troubleshoot data and pipeline-related issues to ensure smooth operation and reliability.
- Migration of DAGs from AWS to GCP by handling version upgrade and architectural changes
- Configuration changes for Hive jobs and spark jobs (Python, Scala) migration based on the cloud environment
- Collaborate with development and operations teams to streamline the deployment process and ensure continuous integration and continuous deployment practices.
- Define and enforce best practices for version control, branching, and code promotion in collaboration with development teams.
- Work closely with cross-functional teams to troubleshoot and resolve deployment issues and optimize release processes.
- Knowledge of LMS and education sector
- Working knowledge of IICS
-
Data Engineer 169368 Full-time
3 days ago
HyreFox Consultants Bengaluru, India Full timeBuild and maintain end-to-end data engineering pipelines on batch data. · - Design and implement batch data ingestion pipelines. · - Design and implement batch data transformation pipelines. · - Design and implement batch data storage solutions. · - Implement batch data governanc ...
-
Azure Data Engineer-3-10years-hyderabad
19 hours ago
Black & White Business Solutions Bengaluru, India**Job Information**: · Industry · **IT Services** · *** · Province · **Karnataka** · *** · City · **Bangalore North** · *** · Postal Code · **560001** · *** · Country · **India** · Job Title:Azure data Bricks · Years of EXP:3 to 10Years · Location:Hyderabad · NP:30Days/Immediate ...
-
Sr Data Engineer/Data Engineer
3 days ago
Trigent Software - Professional Services Bengaluru, IndiaSenior Data Engineer (5+ Years) / Data Engineer (3+): · Experience working with distributed technology tools for developing Batch and Streaming pipelines using · SQL, Spark, PySpark, · Airflow [3+ years], · Spark with Scala (optional). · Able to write code which is optimized ...
-
Data Engineer
20 hours ago
Joveo Bengaluru, IndiaWe are looking for talented Data Engineers who are passionate about data engineering and have strong experience in building scalable data pipelines. As a Data Engineer at Joveo, you will play a leading role in designing and building datasets that power analytics for teams across ...
-
Data Engineer
16 hours ago
Ascendion Bengaluru, IndiaJob Title: Data Engineer/ Lead · Duration: Full Time · Location: Bengaluru, Hyderabad, Chennai, Pune (Hyrbrid) · NOTE: Looking for Immediate Joiners ( 2-3 weeks notice) · Proficient in Azure Cloud services (or GCP is Ok) for data engineering tasks. · Familiarity with Kubeflow, wi ...
-
Data Engineer
5 days ago
Cloud Raptor Bangalore Urban, IndiaAt Cloud Raptor, we specialise in providing scalable and efficient solutions to businesses across · many industries. Focused on cloud technologies and technical expertise, we help organisations · navigate the ever-changing landscape of the digital era. · With dedicated Centres of ...
-
Engineer, Data
1 day ago
Giant Eagle GCC Karnataka, IndiaJob Summary · As a Data Engineer on the Customer Loyalty & Experience team, you will be working on a team to bring customer-centric personalization to life. In this role, you will be empowered to develop data solutions in support of marketing analytics and data science by engagin ...
-
Data Engineering
1 day ago
Fractal Bengaluru, IndiaWe are seeking a talented Lead Data Engineering Manager to support a Fortune 500 Technology Client. The ideal candidate will have 7-12 years of experience in setting up & implementing digital analytics (Adobe Analytics). leading data modeling and ETL development, APIs that serve ...
-
Data Engineer
5 days ago
ParallelDots Bengaluru, IndiaCompany Description · ParallelDots is a leading image recognition solution provider for FMCG companies and retailers globally. Our technology helps optimize in-store execution and maximize sales. For more information on our technology, visit · Role Description · This is a full-t ...
-
Data Engineer
1 day ago
Informatica Bengaluru, IndiaBuild Your Career at Informatica · We're looking for a diverse group of collaborators who believe data has the power to improve society. Adventurous minds who value solving some of the world's most challenging problems. Here, employees are encouraged to push their boldest ideas f ...
-
Data Engineers
1 day ago
Multplyr Bengaluru, IndiaPayroll Company: Multplyr · Role: Data Engineers · Experience: years, across levels · Location: Hybrid / WFO at our client offices in Bengaluru, Mumbai, Pune, Delhi/NCR, Hyderabad and Chennai · Type: Contract / Contract To Hire · Typical Duration: months · About Us · Multplyr is ...
-
Data Engineer
20 hours ago
UNO Digital Bank Bengaluru, IndiaOverview: · We are looking for an experienced data engineer to join our team. You will use various methods to transform raw data into useful data systems. Overall, you'll strive for efficiency by aligning data systems with business goals. · To succeed in this data engineering po ...
-
Data Engineer
1 day ago
Banyan Cloud Bengaluru, IndiaWho are we? · About US · Banyan Cloud is a US-based Cyber Security Product Development Company, headquartered in San Jose, California, USA, which owns the SaaS product "Banyan Cloud", First of its kind Cyber Security Platform, which simplifies the code to cloud security for multi ...
-
Data Engineer
5 days ago
TEKsystems Bengaluru, IndiaJob Description: · We are seeking a skilled and motivated individual to join our team as a Snowflake Data Engineer with expertise in Python development. As a key member of our data team, you will play a crucial role in designing, implementing, and optimizing data pipelines withi ...
-
Data Engineer
2 days ago
Informatica Bengaluru, IndiaBuild Your Career at Informatica · We're looking for a diverse group of collaborators who believe data has the power to improve society. Adventurous minds who value solving some of the world's most challenging problems. Here, employees are encouraged to push their boldest ideas f ...
-
Data Engineer
1 day ago
Indium Software Bengaluru, IndiaTitle: Data Engineer · Experience: 2-6/7 Years · Work Location: Bengaluru/Hyderabad · Work Mode: Work from Office · Skills: Adv SQL, python, Spark or pyspark, Hive, Airflow, ETL, Good Communication skills, Business acumen , Team handling experience. · Roles and Responsibilities: ...
-
Data Engineer
16 hours ago
Demyst Bengaluru, IndiaOur Solution · Demyst unlocks innovation with the power of data. Our platform helps enterprises solve strategic use cases, including lending, risk, digital origination, and automation, by harnessing the power and agility of the external data universe. We are known for harnessing ...
-
Data Engineer
5 days ago
TekWissen India Bengaluru, IndiaRole: Data Engineer · Location: Bangalore, India · Total experience: 7+ Years · Relevant experience: 5-7 Years · Expected Notice period: Immediate · POSITION SUMMARY · Data Engineers build and maintain data systems in support of data analytics and data science activities. The Dat ...
-
Data Engineer
5 days ago
Razorpay Bengaluru, IndiaThe Role: · We are looking for a savvy Data Engineer to join our growing team, responsible for expanding and optimizing our data and data pipeline architecture, as well as optimizing data flow. The ideal candidate is an experienced data pipeline builder and data wrangler who enjo ...
-
Data Engineer
20 hours ago
IRIS Software Group Bengaluru, IndiaAbout IRIS : · IRIS Software Group is one of the UK's largest privately held software companies. Its purpose is to be the most trusted provider of mission-critical software and services, ensuring customers get it right first time, every time. · IRIS takes the pain out of processe ...
Senior Data Engineer - Bangalore Urban, India - Quantiphi
Description
Role & Responsibilities:
As a Data Engineer, you will play a pivotal role in designing, developing, and optimizing data pipelines on Google Cloud Platform (GCP). Your primary responsibility will be to ensure efficient data flow and seamless integration with GCP services while adhering to best practices and data architecture standards. Should be comfortable working with extremely abstract and ambiguous requirements. Must be capable of handling extremely changing business environments with the capability to wear multiple hats at once. Must be able to consult improvisations in various
old school data engineering methods. Should be a very good team player.
Skills required:
Good to have: