Lead Data Engineer - Bengaluru / Bangalore - confidential

    confidential
    confidential Bengaluru / Bangalore

    3 days ago

    Full time
    Description

    Lead Databricks Data Engineer

    • Ecolab is looking for a Data Engineer to be part of a dynamic team that's at the forefront of technological innovation. We're leveraging cutting-edge AI to create novel solutions that optimize operations for our clients, particularly within the restaurant industry. Our work is transforming how restaurants operate, making them more efficient and sustainable.
    • As a key player in our new division, you'll have the unique opportunity to help shape its culture and direction. Your contributions will directly impact the success of our innovative projects and help define the future of our product offerings.Additionally, you will experience the best of both worlds with this team at Ecolab: the agility and creativity of a startup paired with the stability and resources of a global leader. Our collaborative environment fosters innovation while providing the support and security you need to thrive.

    Responsibilities

    • Design, develop, and maintain scalable and robust data pipelines on Databricks (Spark SQL, PySpark, Delta Lake).
    • Collaborate with data scientists and analysts to understand data requirements and deliver solutions.
    • Optimize and troubleshoot existing data pipelines for performance and reliability.
    • Ensure data quality and integrity across various data sources.
    • Implement data security and compliance best practices.
    • Monitor data pipeline performance, implement data quality checks, and conduct necessary maintenance and updates.
    • Document data pipeline processes and technical specifications.
    • Implement robust pipeline orchestration using tools like Databricks Workflows, dbt, or similar.
    • Generate and maintain data quality reports and dashboards.
    • Implement Infrastructure as Code (IaC) principles for managing Databricks infrastructure.

    Minimum Qualifications

    • Bachelors degree and 8 years work experience; or no degree and 12 years combined education and equivalent work experience.
    • 3 years of experience (work or educational) with a data engineering focus.
    • Proven experience in Databricks (Delta Lake, Workflows, Asset Bundles).
    • Proven experience in distributed data processing technologies (Spark SQL, PySpark).
    • Strong knowledge in designing and developing ETL pipelines.
    • Experience with data quality monitoring and reporting.
    • Experience working in a collaborative environment with data scientists and software engineers.

    Preferred Qualifications

    • Masters degree (MS) in Computer Science or related engineering field.
    • Proficiency in Databricks (Delta Lake, Workflows, Asset Bundles).
    • Proficiency in distributed data processing technologies (Spark SQL, PySpark).
    • Experience with pipeline orchestration tools (Databricks Workflows, dbt, etc.).
    • Experience with data visualization tools (e.g., Tableau, Power BI).
    • Working experience with machine learning platforms and tools.
    • Experience with real-time data streaming technologies (e.g., Kafka, Kinesis).

  • Data Engineer

    1 week ago

    Only for registered members Bengaluru, Chennai, Hyderabad

    Sviluppare pipeline dati utilizzando Python e SQL. · ...

  • Data Engineer, IN Data Engineering

    4 days ago

    Only for registered members Bengaluru, Karnataka

    The Amazon IN Platform Development team is looking to hire a rock star Data/BI Engineer to build for pan Amazon India businesses. · ...

  • Data Engineer, IN Data Engineering

    4 hours ago

    Only for registered members Bengaluru

    Design and implement data infrastructure for analytics needs. · Interface with other teams to extract and load data from various sources using SQL and AWS technologies. · ...

  • Data Engineer, IN Data Engineering

    4 days ago

    Only for registered members Bengaluru

    Amazon IN Platform Development team is looking to hire a rock star Data/BI Engineer to build for pan Amazon India businesses. · ...

  • Data Engineer, IN Data Engineering

    4 days ago

    Only for registered members Bengaluru Full time

    +Job summary · A rock star Data/BI Engineer to build for pan Amazon India businesses. · +Responsibilities · Design, implement and support an data infrastructure for analytics needs of large organization,Interface with other technology teams to extract, transform, and load data fr ...

  • Data Engineer, IN Data Engineering

    4 days ago

    Only for registered members Bengaluru, Karnataka

    We are looking to hire a rock star Data/BI Engineer to build for pan Amazon India businesses. · The BI Engineer will play a key role in contributing to the success of each focus area by partnering with respective business owners and leveraging data to identify areas of improvemen ...

  • Data Engineer for Data

    2 days ago

    Only for registered members Bengaluru Full time

    We invite you to join our digital transformation journey. In this role, you will have the opportunity to boost your skills and become the heart of an innovative engine that is contributing to global impact and improvement. · ...

  • Staff Data Engineer, Data Engineering

    1 month ago

    Only for registered members Bengaluru, INDIA, India

    Unlocking the value available through the innovative use of data on behalf of consumers, businesses, and communities is key to our future. With our ongoing commitment to Visa's Data Values and the responsible use of data, we at Visa have a bold vision to continue to grow and acce ...

  • Staff Data Engineer, Data Engineering

    1 month ago

    Only for registered members Bengaluru, Karnataka

    We are a Data Engineering team within GDO, responsible for designing, building, and optimizing data pipelines and platforms that power Visa's analytics. · ...

  • Staff Data Engineer, Data Engineering

    1 month ago

    Only for registered members Bengaluru

    We are seeking enthusiastic leaders who are change-agents and passionate about the future of data around the world to join our Data Engineering team within GDO. You will play a critical role in designing and delivering scalable, high-quality data solutions that enable intelligent ...

  • Data Engineer, Data Engineering Solutions

    3 weeks ago

    Only for registered members Bengaluru, Karnataka

    We are experts in data working to make it cost-effective trustworthy and understandable. We build pipelines processing billions of events daily and are stewards of canonical data warehouses and datasets delivering products for Stripe Users while embedding with teams to build thei ...

  • Staff Data Engineer, Data Engineering

    1 month ago

    Only for registered members Bengaluru

    + Design and deliver scalable, high-quality data solutions that enable intelligent decision-making across Global Data Engineering capabilities · + Build and utilize advanced data modeling frameworks and leverage modern architectural patterns to design scalable, well-structured, a ...

  • Data Engineer, Data Engineering Solutions

    3 weeks ago

    Only for registered members Bengaluru Full time

    We're looking for a manager to drive the Data Engineering Solutions Team in solving high-impact,cutting-edge data problems. · Deliver cutting-edge data pipelines that scale to users' needs,focusing on reliability and efficiency. · ...

  • Data Engineer, Data Engineering Solutions

    3 weeks ago

    Only for registered members Bengaluru

    Who we are · About Stripe · Stripe is a financial infrastructure platform for businesses.Millions of companies—at the world's largest enterprises to the most ambitious startups—use Stripe to accept payments, grow their revenue, and accelerate new business opportunities. · ...

  • Data Engineer

    1 week ago

    Only for registered members Bangalore, Karnataka, India

    We are seeking a highly skilled AWS Data Engineer to join our dynamic team. · As an AWS Data Engineer, you will be responsible for designing, building and maintaining data pipelines and ETL processes on the AWS platform. · In addition to your technical expertise we value individ ...

  • Data Engineer

    6 days ago

    Only for registered members Bengaluru

    As a Data Engineer you will work on building testing and maintaining data pipelines and connectors that integrate data from various sources APIs Databases Files SaaS applications into the data platform. · ...

  • Data Engineer

    3 days ago

    confidential Bengaluru / Bangalore Full time

    We are seeking a skilled Data Engineer to join our team in India. · ...

  • Staff Data Engineer, Data Engineering

    1 month ago

    Only for registered members Bengaluru Full time

    +We are a Data Engineering team within GDO, responsible for designing, building, and optimizing data pipelines and platforms that power Visa's analytics, and decisioning capabilities. Our focus is on data quality, scalability, and reliability, enabling seamless integration of com ...

  • Data Engineer

    1 month ago

    Only for registered members Bangalore

    We are looking for an experienced data engineer to join our team. · ...

  • Data Engineer

    2 days ago

    Only for registered members Bengaluru, Karnataka

    AuxoAI is seeking a skilled and experienced Data Engineer to join our dynamic team. · ...

  • Data Engineer

    4 days ago

    Only for registered members Bangalore

    We are looking for an experienced Lead Data Engineer to drive large-scale data platform initiatives in a product-based environment. · - Lead design and development of scalable data pipelines and distributed systems · - Build and optimize big data solutions using Spark (Scala/PySp ...

Jobs
>
Lead data engineer
>
Jobs for Lead data engineer in Bengaluru