Big Data Engineer - Bengaluru / Bangalore - confidential

    confidential
    confidential Bengaluru / Bangalore

    4 days ago

    Full time ₹900,000 - ₹2,800,000 (INR) per year *
    Description

    We are actively seeking a skilled Big Data Engineer to join our client's dynamic team through Acme Services. This pivotal role requires solid experience in ETL / Data Engineering / Data Warehousing principles. The ideal candidate will possess strong proficiency in Python, PySpark, and SQL or other ETL tools, along with hands-on experience in various database and big data technologies such as MySQL, SQL Server, MongoDB, Hadoop, Hive, Pig, and Spark. Experience with orchestration tools like AirFlow and container platforms like OpenShift would be a significant advantage.

    Key Responsibilities

    • Data Pipeline Development: Design, build, and maintain robust and scalable ETL (Extract, Transform, Load) pipelines to ingest, process, and transform large datasets from various sources into data warehouses or data lakes.
    • Big Data Technologies: Work extensively with Big Data platforms including Hadoop, Hive, Pig, and Spark, leveraging their capabilities for data processing and analytics.
    • Programming & Scripting: Utilize Python, PySpark, and SQL for data manipulation, scripting, and developing data engineering solutions.
    • Database Management: Interact with and manage data in various database systems such as MySQL, SQL Server, and MongoDB.
    • Data Warehousing: Contribute to the design, implementation, and optimization of Data Warehouses, ensuring data quality and accessibility for business intelligence and reporting.
    • Orchestration & Automation (Advantage): (If applicable) Use AirFlow for scheduling, monitoring, and orchestrating complex data workflows.
    • Containerization (Advantage): (If applicable) Leverage OpenShift for deploying and managing data applications in a containerized environment.
    • Troubleshooting & Optimization: Identify and resolve data-related issues, and optimize data processes for performance and efficiency.

    Skills

    • Solid experience in ETL / Data Engineering / Data Warehouse.
    • Strong proficiency in Python, PySpark, and SQL or other ETL tools.
    • Hands-on experience with databases including MySQL, SQL Server, MongoDB.
    • Expertise in Hadoop, Hive, Pig, and Spark.
    • Experience in AirFlow (big advantage).
    • Experience in OpenShift (big advantage).
    • Strong analytical and problem-solving skills.
    • Ability to work with large, complex datasets.

    Qualifications

    • Proven professional experience as a Big Data Engineer or in a similar data-focused role.
    • Demonstrable experience in building and managing data pipelines.
    • Bachelor's degree in Computer Science, Engineering, or a related field, or equivalent practical experience.
    * This salary range is an estimation made by beBee
  • Work in company

    Data Engineer

    Only for registered members

    Wir suchen einen DevOps-und Data Engineering Experten mit 7-12 Jahren Erfahrung. · Kubeflow Thingsboard Apache Superset Dremio usw. · ...

    Bengaluru, Karnataka

    3 weeks ago

  • Work in company

    Data Engineer

    Only for registered members

    Weiterhin suchen wir einen DevOps-Experten mit 7 bis 12 Jahren Erfahrung in relevanten Technologien, Konfiguration und Wartung von Kubernetes-Umgebungen einschließlich GitOps (ArgoCD), Ressourcen-Templating mit Helmcharts und Kustomize · ...

    Bengaluru

    3 weeks ago

  • Work in company

    Data Engineer

    Only for registered members

    Sviluppare pipeline dati utilizzando Python e SQL. · ...

    Bengaluru, Chennai, Hyderabad

    2 weeks ago

  • Work in company

    Data Engineer, IN Data Engineering

    Only for registered members

    +Job summary · A rock star Data/BI Engineer to build for pan Amazon India businesses. · +Responsibilities · Design, implement and support an data infrastructure for analytics needs of large organization,Interface with other technology teams to extract, transform, and load data fr ...

    Bengaluru Full time

    2 weeks ago

  • Work in company

    Data Engineer, IN Data Engineering

    Only for registered members

    The Amazon IN Platform Development team is looking to hire a rock star Data/BI Engineer to build for pan Amazon India businesses. · ...

    Bengaluru, Karnataka

    1 week ago

  • Work in company

    Data Engineer, IN Data Engineering

    Only for registered members

    Amazon IN Platform Development team is looking to hire a rock star Data/BI Engineer to build for pan Amazon India businesses. · ...

    Bengaluru

    2 weeks ago

  • Work in company

    Data Engineer, IN Data Engineering

    Only for registered members

    Design and implement data infrastructure for analytics needs. · Interface with other teams to extract and load data from various sources using SQL and AWS technologies. · ...

    Bengaluru

    1 week ago

  • Work in company

    Data Engineer Data Engineer

    Only for registered members

    · At ANZ, we're applying new ways technology and data can be harnessed as we work towards a common goal: to improve the financial wellbeing and sustainability of our millions of customers. · ...

    Bengaluru Full time

    1 week ago

  • Work in company

    Data Engineer, IN Data Engineering

    Only for registered members

    We are looking to hire a rock star Data/BI Engineer to build for pan Amazon India businesses. · The BI Engineer will play a key role in contributing to the success of each focus area by partnering with respective business owners and leveraging data to identify areas of improvemen ...

    Bengaluru, Karnataka

    1 week ago

  • Work in company

    Data Engineer for Data

    Only for registered members

    We invite you to join our digital transformation journey. In this role, you will have the opportunity to boost your skills and become the heart of an innovative engine that is contributing to global impact and improvement. · ...

    Bengaluru Full time

    1 week ago

  • Work in company

    Data Engineer/Senior Data Engineer

    Only for registered members

    Requirements · Job Title: Data Engineer – II / Senior Data Engineer · Job Location: Bengaluru, India · Job Summary: We are a leading Software as a Service (SaaS) company that specializes in the transformation of data in the US healthcare industry through cutting-edge Artificial I ...

    Bengaluru

    5 days ago

  • Work in company

    Data Engineer, senior data engineer

    Only for registered members

    Data Engineer role involves participating in the software development lifecycle for enterprise data projects, designing & maintaining robust ETL processes & data pipelines using modern platforms. · ...

    Bengaluru, Hyderabad

    5 days ago

  • Work in company

    Data Engineer, Data Engineering Solutions

    Only for registered members

    We are experts in data working to make it cost-effective trustworthy and understandable. We build pipelines processing billions of events daily and are stewards of canonical data warehouses and datasets delivering products for Stripe Users while embedding with teams to build thei ...

    Bengaluru, Karnataka

    1 month ago

  • Work in company

    Data Engineer, Data Engineering Solutions

    Only for registered members

    We're looking for a manager to drive the Data Engineering Solutions Team in solving high-impact,cutting-edge data problems. · Deliver cutting-edge data pipelines that scale to users' needs,focusing on reliability and efficiency. · ...

    Bengaluru Full time

    1 month ago

  • Work in company

    Senior Data Engineer

    Only for registered members

    Safran est un groupe international de haute technologie opérant dans les domaines de l'aéronautique (propulsion, équipements et intérieurs), de l'espace et de la défense. · ...

    Bengaluru, Karnataka ₹10 - ₹15 (INR) per year

    5 days ago

  • Work in company

    Data Engineer

    Only for registered members

    As a Data Engineer you will work on building testing and maintaining data pipelines and connectors that integrate data from various sources APIs Databases Files SaaS applications into the data platform. · ...

    Bengaluru

    2 weeks ago

  • Work in company

    Data Engineer

    Bajaj Technology Services

    Role Summary · The Data Engineer will design, build, and optimize large-scale data pipelines and data platforms. This role is hands-on and plays a key part in translating architectural designs into production-ready solutions. · Key Responsibilities · Build and maintain scalable b ...

    Bengaluru ₹900,000 - ₹2,500,000 (INR) per year

    3 days ago

  • Work in company

    Data Engineer

    Only for registered members

    We are seeking a Sr. Data Engineer to join our Data Engineering team within our Enterprise Data Insights organization to build data solutions... · ...

    Bengaluru, Delhi, Mumbai, NCR

    1 month ago

  • Work in company

    Data Engineer

    Only for registered members

    About the Job · Skills · data lakehouse · python · etl pipelines · Data Engineering · apache hop · Databases · NoSQL · SQL · Location: Bangalore (5 days work from office, HSR Layout) · Job Type: Full Time · Industry: Agritech · Openings: Multiple- junior, mid level etc · Python- ...

    Bangalore ₹900,000 - ₹2,500,000 (INR) per year Full time

    42 minutes ago

  • Work in company

    Data Engineer

    Only for registered members

    Required Experience - AWS, ETL, Data Modelling, Data Integration, Data Manipulation, GITHUB, Azure Devops, Databrick, Good to have pytest · We are seeking a highly skilled AWS Data Engineer to join our dynamic team. As an AWS Data Engineer, you will be responsible for designing, ...

    Bangalore, Karnataka, India ₹900,000 - ₹2,500,000 (INR) per year

    15 hours ago

  • Work in company

    Senior Data Engineer

    Only for registered members

    Le rôle consiste à concevoir et mettre en œuvre des solutions de lac de données et lakehouse scalables sur AWS pour les teas & analytics workloads. · AWS Core Services : S3, Lambda, DynamoDB, · RDS (PostgreSQL), API Gateway, CloudFormation, · CloudWatch. · Data Engineering Tools ...

    Bengaluru

    6 days ago

Jobs
>
Big data engineer
>
Jobs for Big data engineer in Bengaluru