Jobs
>
Delhi

    Big Data Engineer - Any Location/Bangalore, India - Careerfit

    Careerfit
    Careerfit Any Location/Bangalore, India

    Found in: Talent IN 2A C2 - 4 days ago

    Default job background
    permanent Technology / Internet
    Description

    Responsibilities :

    Solution Design :

    • Design, develop, and maintain scalable big data solutions and data pipelines for efficient data ingestion, processing, storage, and analysis.
    • Collaborate with data scientists, analysts, and business stakeholders to identify data requirements, use cases, and business objectives.
    • Translate business needs into technical designs and select appropriate big data technologies for optimal performance.

    Stakeholder Collaboration :

    • Partner with data scientists, analysts, and business stakeholders to understand data needs, answer technical questions, and provide data access solutions.
    • Communicate complex technical concepts clearly and concisely to nontechnical audiences.
    • Foster strong working relationships across departments to ensure successful datadriven projects.

    Data Processing :

    • Implement and optimize data ingestion pipelines using tools like Apache Kafka to acquire data from various sources in realtime or batch mode.
    • Utilize big data technologies such as Apache Hadoop, Spark, and HBase to efficiently process and store large datasets.
    • Develop and maintain data quality checks and data cleansing routines to ensure data integrity and consistency.

    Data Transformation :

    • Design and maintain data models and schemas for structured and unstructured data.
    • Develop and implement ETL (Extract, Transform, Load) processes to transform raw data into usable formats for analysis.
    • Perform data cleansing, normalization, and deduplication tasks to improve data quality for downstream applications.

    Performance Optimization :

    • Monitor the performance, scalability, and reliability of big data solutions.
    • Identify and troubleshoot performance bottlenecks to ensure efficient data processing and access.
    • Continuously optimize data pipelines and infrastructure to meet evolving data volumes and processing demands.

    Data Governance :

    • Champion data governance best practices throughout the data lifecycle.
    • Implement robust data security measures to safeguard sensitive information.
    • Ensure adherence to data privacy regulations and compliance requirements.

    Continuous Learning :

    • Stay uptodate with emerging big data technologies, trends, and best practices through ongoing learning initiatives.
    • Participate in professional development activities, such as conferences and workshops, to enhance your technical skillset.

    Requirements :

    • Bachelor's or Master's degree in Computer Science, Information Systems, or a related field.
    • Proven experience in big data engineering or related roles (minimum X years).
    • Proficiency in programming languages like Java, Python, or Scala.
    • Solid understanding of distributed computing frameworks (Hadoop, Spark).
    • Experience with data modeling, schema design, and ETL processes.
    • Excellent problemsolving, analytical, and communication skills.
    • Ability to work effectively in a team environment.

    Skills :

    • Big Data Engineering : Proficient in designing, developing, and maintaining big data solutions and pipelines.
    • Data Processing : Skilled in implementing and optimizing data ingestion, processing, and storage using big data technologies.
    • Collaboration : Ability to collaborate effectively with crossfunctional teams.
    • Problem-Solving : Strong problemsolving skills to identify and resolve technical challenges.

    Preferred Qualifications :

    • Certification in big data technologies (Cloudera Certified Professional, Hortonworks Certified Developer).
    • Experience with cloud platforms (AWS, Azure, GCP) for big data deployments.
    • Experience with data visualization tools (Tableau, Power BI).
    • Experience with scripting languages (Bash, Shell).)
  • KARYA CONSULTANTS PRIVATE LIMITED

    Data Engineer

    Found in: Talent IN 2A C2 - 3 days ago

    KARYA CONSULTANTS PRIVATE LIMITED Delhi, India

    As a GCP Data Engineer, you will be a key player in designing, implementing, and optimizing data solutions on the Google Cloud Platform. · We are seeking a professional with hands-on experience in data engineering and a strong understanding of GCP services. · Key Responsibilities ...

  • Vision Beyond Resources India Pvt. Ltd.

    Data Engineer

    Found in: Talent IN 2A C2 - 3 days ago

    Vision Beyond Resources India Pvt. Ltd. Noida/Gurgaon/Gurugram/Delhi NCR/Chennai/Mumbai, India permanent

    Role : Data : UK Shift Range : 7+ Requirements : · - Excellent understanding of data structures with 5+ years of experience in enterprise-wide data platforms design and implementation. · - Strong knowledge of common design patterns used in Data Integrations and Warehousing. · - P ...

  • WovV Technologies

    Data Engineer

    Found in: Talent IN 2A C2 - 3 days ago

    WovV Technologies Bangalore/Any Location, India permanent

    Job Description : Data Engineer · Location : Remote · Experience : 5+ years. · Durations : 6 months with a very likely extension · Mandatory Technology : DBT, SQL, Snowflake,data integration practices, and Tableau · Primary Skills : Expertise in transaction refunds and a comprehe ...

  • Vbeyond

    Data Engineer

    Found in: Talent IN 2A C2 - 3 days ago

    Vbeyond Delhi NCR/Mumbai/Noida, India permanent

    JOB SUMMARY : · We are seeking a skilled and experienced Data Engineer with strong developments skills and solid track record in terms of implementing scalability, availability and excellence when building data pipelines and platforms for our royalties and revenue share initiati ...

  • INSIGHT GLOBAL SOLUTIONS

    Data Engineer

    Found in: Talent IN 2A C2 - 3 days ago

    INSIGHT GLOBAL SOLUTIONS Bangalore/Any Location, India permanent

    Data Engineer · Must Haves : · - Bachelors Degree in science, technology, or another related field. · - 3+ years of data engineering experience. · - 3+ years of experience with Cloud platforms like GCP, Azure, Airflow. · - 4+ years of data warehouse, relational database, and SQL ...

  • HS Solutions Inc

    Data Engineer

    Found in: Talent IN 2A C2 - 3 days ago

    HS Solutions Inc Any Location/Overseas/International/Bangalore/Atlanta, India permanent

    About the job : · Tech stack : · BigQuery , any ETL tool (Informatica, Talend, DataStage), Dataflow, Dataproc. · - 3-5 years' Experience in Data warehouse and Data lake implementation. · - 1-2 years of experience in Google Cloud Platform (especially Big Query). · - 1-2 years of w ...

  • SKYGATE CONSULTING

    Data Engineer

    Found in: Talent IN 2A C2 - 3 days ago

    SKYGATE CONSULTING Bangalore/Gurgaon/Gurugram/Delhi NCR, India permanent

    Data Engineer (Snowflake) · Job Description : · As part of the team, you will be responsible for building and running the data pipelines and services that are required to support business functions/reports/dashboard.. We are heavily dependent on BigQuery/Snowflake, Airflow, Stitc ...

  • Resources valley

    Data Engineer

    Found in: Talent IN 2A C2 - 4 days ago

    Resources valley Jaipur/Remote, India permanent

    Job Description : · Ownership of Data Engineering solution development, including ingestion, transformation, loading, modelling, curation, profiling, migration, and validation of data, also maintaining security, access control, compliance, and SLAs · - Design and create data mode ...

  • USEReady Technology Pvt. Ltd.

    Data Engineer

    Found in: Talent IN 2A C2 - 4 days ago

    USEReady Technology Pvt. Ltd. Any Location/Bangalore, India permanent

    Role Responsibility : · - Translate functional specifications and change requests into technical specifications. · - Translate business requirement document, functional specification, and technical specification to related coding. · - Develop efficient code with unit testing and ...

  • Triumph Ladders Pvt. Ltd.

    Data Engineer

    Found in: Talent IN 2A C2 - 4 days ago

    Triumph Ladders Pvt. Ltd. Delhi, India permanent

    Title - Data Engineer · Exp years · Location - Delhi · Notice period days · Responsibilities : · - Collaborate with Data Scientists to test and scale new algorithms through pilots and later industrialize · the solutions at scale to the comprehensive fashion network of the Group ...

  • Triumph Ladders Pvt. Ltd.

    Data Engineer

    Found in: Talent IN 2A C2 - 4 days ago

    Triumph Ladders Pvt. Ltd. Delhi, India permanent

    Title - Data Engineer · Exp yrs · Location - Delhi · Notice period days · Responsibilities : · - Collaborate with Data Scientists to test and scale new algorithms through pilots and later industrialize · the solutions at scale to the comprehensive fashion network of the Group · - ...

  • Woolworths Group International

    Data Engineering Manager

    Found in: Talent IN 2A C2 - 1 day ago

    Woolworths Group International Delhi, India

    What we're like · At BIG W every day's a big day. For you, this could be the start of something big. · There are over 15,000 of us, in the biggest cities and the tiniest towns. And each of us has an important role to play to help make a real difference for families by providing r ...

  • Muoro

    Senior Data Engineer

    Found in: Talent IN 2A C2 - 4 days ago

    Muoro Any Location, India permanent

    Role & Responsibilities: · - Design, architect, and develop code in a testable, scalable, and performant manner. · - Work closely with Product Managers, Cross Product Engineers and Data Engineers to optimize and deliver innovative solutions. · - Build a team that mines, analyzes, ...

  • Visionyle

    Azure Data Engineer

    Found in: Talent IN 2A C2 - 4 days ago

    Visionyle Any Location, India permanent

    Position Name: Azure Data Engineer · Position Mode: Permanent · Experience: 6 Years · Joining Location: Remote · Notice: Immediate · Mode of Interview : 2-3 rounds · SUMMARY: · Azure Data Engineer should have experience on Azure data lake, Azure SQL database, Databricks, Azure Da ...

  • Randstad

    Senior Data Engineer

    Found in: Talent IN 2A C2 - 4 days ago

    Randstad Bangalore/Any Location, India permanent

    Job Description: · We are looking for a highly skilled Senior Data Engineer with over 5 years of experience in the data engineering field. · The ideal candidate will have strong expertise in PySpark, SQL databases, and Big Data programming for data transformation and aggregations ...

  • Sureminds Solutions Pvt. Ltd.

    Big Data Engineer

    Found in: Talent IN 2A C2 - 3 days ago

    Sureminds Solutions Pvt. Ltd. Bangalore/Delhi NCR/Noida/Gurgaon/Gurugram/Pune, India permanent

    Job Description have skills : · 1. Handson experience of Spark, Scala and conversant with SQL · 2. Good knowledge on Hadoop (Oozie) · 3. Reverse engineer the SQL queries, Scala code to understand functionality · 4. Capable of identifying, analysing and interpret patterns and tren ...

  • INSIGHT GLOBAL SOLUTIONS

    Data Warehouse Engineer

    Found in: Talent IN 2A C2 - 3 days ago

    INSIGHT GLOBAL SOLUTIONS Any Location/Bangalore, India permanent

    Must-haves : · - 5+ years of Data Warehouse engineering experience. · - Strong proficiency in Linux/Unix tools and scripting (Python). · - Strong proficiency with integrating ETL tools with database load utilities. · - Cloud Data Lake/Data warehouse technology and tools (Databric ...

  • Impetus

    GCP data Engineer

    Found in: Talent IN 2A C2 - 2 days ago

    Impetus Delhi, India

    The ideal candidate is a self-motivated, multi-tasker, and demonstrated team-player. You will be a lead developer responsible for the development of new software products and enhancements to existing products. You should excel in working with large-scale applications and framewor ...

  • Geektrust

    Senior Data Engineer

    Found in: Talent IN 2A C2 - 3 days ago

    Geektrust Bangalore/Remote, India permanent

    Role : Senior Data Engineer · Focus : AWS ecosystem · Key Responsibilities : Designing, developing, and maintaining data pipelines; ensuring the reliability, efficiency, and scalability of data infrastructure; collaborating with cross-functional teams. · Primary Skills : · Prog ...

  • Change leaders

    AWS Data Engineer

    Found in: Talent IN 2A C2 - 3 days ago

    Change leaders Pune/Chennai/Mumbai/Bangalore/Delhi NCR/Hyderabad, India permanent

    Job Description : AWS Glue + Python Pyspark · Job Overview : · Candidate should be responsible for the configuration, administration, and optimization of the AWS Glue + Python Pyspark platform. This role involves ensuring the smooth operation of AWS Glue + Python Pyspark for data ...