Jobs
>
Bengaluru

    Lead ETL Engineer - Bengaluru, India - YITRO BUSINESS CONSULTANTS (INDIA) PRIVATE LIMITED

    YITRO BUSINESS CONSULTANTS (INDIA) PRIVATE LIMITED
    YITRO BUSINESS CONSULTANTS (INDIA) PRIVATE LIMITED Bengaluru, India

    1 week ago

    Default job background
    Description

    Role :
    Lead ETL Feature Engineer

    Years of Experience : 5 to 9 years


    Job Location :
    Bangalore


    Mode of Work :
    Work from office


    Shift :
    General


    Job Type :
    Permanent


    Responsibilities :

    • Develop and maintain ETL pipelines to ingest and process large-scale datasets
    • Develop a Python connector designed for ETL applications, enabling the collection of historical samples and the generation of intelligent samples for AI/ML tasks
    • Demonstrate hands-on experience on Apache Kafka, Amazon Kinesis, and AWS Glue
    • ETL pipeline for AI/ML workload integrations and executions
    • Implement and maintain ETL pipeline and deep understanding of orchestration, scaling, and resource management.
    • Develop ETL samples to manage unstructured data tasks, covering data preprocessing for various types, including emails, Office 365 documents, pictures, voice recordings, videos, and PDFs.
    • Establish a solid understanding and practical knowledge of SQL databases, with a focus on query performance optimization and effective index management.
    • Implement an ETL pipeline with multiple databases to extract samples from both NoSQL and SQL
    • Work seamlessly in a multi-cloud environment, specifically AWS and Azure.
    • Execute hands-on development and deployment of microservices and containerized applications.

    Requirements :

    • Minimum 4 years hands-on experience in setting up ETL & feature engineering part of the data pipelines on cloud or big data ecosystems.
    • Strong hands-on experience in Apache Kafka, Amazon Kinesis, and AWS Glue
    • Holds expert-level knowledge in at least one ETL tool (ETL/ Workflow - SeaTunnel/ Dolphin Scheduler, SSIS, Informatica, Talend, etc.)
    • Experience with big data technologies such as Hadoop, Spark, and Hive
    • Experience in NoSQL and SQL databases (e.g., MongoDB, SnowFlake, PostgreSQL, DeltaLake, Parquet).
    • Strong programming skills in Python, and experience with data manipulation libraries such as pandas, numpy, etc.
    • Handling unstructured data tasks, including preprocessing for various data types such as Mail, Office 365, Pictures, Voice, Video, and PDF
    • Possesses SQL knowledge including query performance tuning, index maintenance and an understanding of database structure
    • Good to have knowledge in Apache Spark, Apache NiFi, Apache Kafka, Apache Airflow, Apache Beam, KubeFlow, Apache Beam and TensorFlow Transform
    • Expertise in networking, security, and cloud platforms (AWS, Azure, GCP, etc.)
    Interested candidate apply here or share your updated resume to
    )
  • DigiKey Global Capability Center

    ETL Support Engineer

    6 days ago

    DigiKey Global Capability Center Bangalore Urban, India

    Organization: Digi-Key Global Capability Center - SMC Square · Location: Bangalore · Experience: 2 to 7 years · Position: ETL Support Engineer - (Level 1) · Responsibilities · Must be able to... · Provide ongoing support for the existing ETL processes, procedures, as well as the ...

  • RS Consultants

    ETL/Databricks Engineer

    3 hours ago

    RS Consultants Bengaluru, India

    ETL (Databricks) · Job description : · Responsibilities : · - Build, document, test and launch new data extract, transform and load (ETL) processes for one or more core data sets. · - Collaborate with Business Intelligence team and other stakeholders for data requirements · - Co ...

  • State Street

    ETL Datawarehouse Engineer

    6 hours ago

    State Street Bengaluru, India Full time

    Who we are looking for: · Looking for a seasoned ETL Datawarehouse engineer worked on Snowflake based implementation at large scale. Have experienced on both Azure and AWS cloud primarily on the architecture, design and implementation · As Senior ETL Datawarehouse engineer you w ...

  • State Street

    ETL Automation engineer

    5 days ago

    State Street Bangalore, India OTHER

    Role Summary & Role Description · As the ETL Automation Engineer in the Corporate Finance Technology team you will · Developing and maintaining robust ETL pipelines using PySpark to process large volumes of data efficiently. · Writing and executing automated tests for ETL process ...

  • Codersbrain technology pvt ltd

    ETL Data Engineer

    1 week ago

    Codersbrain technology pvt ltd Bengaluru, India

    Job Description: ETL - Data Design, Develop and maintain ETL pipelines using Pyspark in Azure Databricks using delta tables. · - Create build from Github and release pipeline for Ingestion and Databricks using Azure Devops / Harness · - Monitor Performance of ETL Jobs, resolve an ...

  • SemperFi Solutions

    ETL/Data Engineering Manager

    5 days ago

    SemperFi Solutions Bengaluru, India

    Shift timings : 12pm to 9 PM · Total experience required : 15-20 years. · Job Description : · - Lead and mentor a team of data engineers in designing, developing, and maintaining efficient and reliable ETL pipelines. · - Develop advanced scripts using Python for data extraction, ...

  • ContinuServe

    Manager - ETL and Data Engineering C220

    2 hours ago

    ContinuServe Bengaluru, India Full time

    ContinuServe Softech is a leading professional services, consulting and outsourcing firm that helps mid-market and Fortune 2000 organizations improve service levels and reduce costs within their back-office operations through the implementation and delivery of shared services sol ...

  • Response Informatics

    Gcp (Google Cloud Platform) + SQL

    2 days ago

    Response Informatics Bengaluru, India

    Hello Connections · We have a critical Job opening for one of our prestigious clients. · Job role: · - GCP +SQL · Location: · - Pune/Hyderabad/Bangalore · Notice Period: · - Immediate/15 days · Experience: · - 6 + years relevant · JD: · Senior Data Engineer · **Position Summary** ...

  • Qualitest

    Data/etl Testing

    1 week ago

    Qualitest Bengaluru, India

    · 8+ years of experience · 1. Hands-on in Data/ETL Testing, Data Warehouse, Data Lakes · 2. Proficient in Data Validation tools such as RightData (preferred), ICEDQ (Integrity Check Engine for Data Quality), Informatica, Tosca DI, Datagaps · 3. Develop custom utilities using Pyt ...

  • Qualitest

    9252#1-data/etl Testing

    1 week ago

    Qualitest Bengaluru, India

    8+ years of experience · 1. Hands-on in Data/ETL Testing, Data Warehouse, Data Lakes · 2. Proficient in Data Validation tools such as RightData (preferred), ICEDQ (Integrity Check Engine for Data Quality), Informatica, Tosca DI, Datagaps · 3. Develop custom utilities using Python ...

  • Apexon

    Etl Informatica Architect

    6 days ago

    Apexon Bengaluru, Karnataka, India

    **About Apexon**: · Apexon is a digital-first technology services firm specializing in accelerating business transformation and delivering human-centric digital experiences. We have been meeting customers wherever they are in the digital lifecycle and helping them outperform thei ...

  • IBST Private Limited

    Etl Informatica Powercentre with Mdm

    2 days ago

    IBST Private Limited Bengaluru, India

    SQL · 5+ years of experience at Enterprise-level ETL development and ETL architecture using Informatica and understanding the COBOL file structures. · Strong experience in MDM · Strong Database knowledge in the areas of general SQL and Oracle · Strong Informatica technical knowle ...

  • Mercedes-Benz Research and Development India Private Limited

    Brake Test Data Analytics

    1 week ago

    Mercedes-Benz Research and Development India Private Limited Bengaluru, India

    **Aufgaben**: · - Education: Bachelors or Masters in Mechanical Engineering, Computer Science, Electrical/Electronic from tier-I universities · - Skill Requirements: · - Expertise in Azure Databricks. Able to connect, load data/files from Azure Data Lake & perform data analytics ...

  • Wipro Limited

    Gen Ai

    1 week ago

    Wipro Limited Bengaluru, Karnataka, India

    **Overview**: · Sr.Data Engineers with minmum 6+ years experinece on Pyhton, SQL, Shell Scriptin, Hive and Spark · ? Expert-level skills in writing and optimizing complex SQL. · ? Knowledge of data warehousing concepts. · ? Experience in data mining, profiling, and analysis. · ? ...

  • Tekfortune India IT Pvt Ltd.

    Solution Data Architect

    1 week ago

    Tekfortune India IT Pvt Ltd. Bengaluru, India

    **Role: Solution Data Architect** · **Experience: 12 to 18 Years** · **Location: Bangalore** · *** · **Notice Period: Immediate to 60 Days** · *** · **Skills Required: AWS/Azure/GCP, Cloud Migration, ETL, Leadership** · *** · **Roles & Responsibilities**: · *** · - Solution Engin ...

  • PradeepIT Consulting Services

    Databricks (Remote)

    4 hours ago

    PradeepIT Consulting Services Bengaluru, Karnataka, India

    **About the job Databricks (Remote)**: · Roles & responsibilities · - Developing Modern Data Warehouse solutions using Databricks and AWS/ Azure Stack · - Ability to provide solutions that are forward-thinking in data engineering and analytics space · - Collaborating with DW/BI l ...

  • Aptra Technologies Pvt Ltd

    Data Enginneer

    1 day ago

    Aptra Technologies Pvt Ltd Bengaluru, Karnataka, India

    Greetings From Ashra Technologies Pvt Ltd · we are Hiring · Job Role:Data Engineer · Experience:6+Years · Locations:Bangalore/Gurgaon · **Focused skills: GCP Services including Big Query, Data flow, Data proc, Apache Beam, Pub Sub, Composer** · **Job description** · - 6 to 9 Year ...

  • Qualitest

    Etl Tester and Database Tester

    1 week ago

    Qualitest Bengaluru, India

    · 1. Database Testing · 2. ETL Testing Concepts · 3. Datawarehouse knowledge - Star Schema, Snowflake Schema, Fact and Dimension Tables · 4. Dataset knowledge · 5. Should be very good on writing medium to complex queries · 6. Dashboard Testing Experience · 7. Tools - JIRA, SQLSe ...

  • Capgemini

    Alteryx 6 to 9 Years Bengaluru, Mumbai

    5 days ago

    Capgemini Bengaluru, India

    Experience on data warehouse, ETL and development of data integration · - Experience in Alteryx data management architecture and design, including Python scripting experience · - Review current Alteryx code, data pipelines and prepare Azure migration strategy with Azure data engi ...

  • VARITE INDIA PRIVATE LIMITED

    Snowflake Architect

    4 days ago

    VARITE INDIA PRIVATE LIMITED Bengaluru, India

    POSITION OVERVIEW : Snowflake Developer \ Data Engineer · - Database Development: Designing, developing, and maintaining databases using Snowflake, a cloud-based data warehousing platform. · - Data MoClienting: Creating and optimizing data models that adhere to industry best prac ...