Data Engineer - New Delhi, India - ZIDEA

    ZIDEA
    ZIDEA New Delhi, India

    1 week ago

    Default job background
    Description

    ResponsibilitiesDesign, build, and maintain scalable data pipelines on AWS using services such as Amazon S3, Glue, and EMR.Develop and optimize ETL processes to ingest data from various sources into our data lake and data warehouse.

    Collaborate with data scientists and analysts to understand data requirements and implement solutions for data transformation, aggregation, and analysis.
    Ensure data quality and integrity by implementing data validation and monitoring processes.
    Optimize performance of data infrastructure and pipelines for efficiency and cost-effectiveness.

    Implement security best practices to protect sensitive data and ensure compliance with regulations such as GDPR and HIPAA.Document design decisions, workflows, and operational procedures to maintain a knowledge base for the team.

    Stay up to date with the latest AWS technologies and industry trends to continuously improve our data infrastructure.


    SkillsProven experience (3+ years) as a Data Engineer, with a focus on AWS cloud technologies, and experience and knowledge on Microsoft Azure technology is a plus.

    Extensive hands-on experience with AWS services such as Amazon Redshift, Amazon EMR, Amazon Kinesis, AWS Glue, etcStrong understanding of data modeling, ETL processes, data warehousing, and data governance principles.

    Familiarity with programming/scripting languages like Python, SQL, and experience with data manipulation and transformation.
    Excellent problem-solving skills with the ability to design and implement creative solutions for complex data challenges.
    Knowledge of data governance and compliance standards.
    Strong communication skills to effectively collaborate with technical and non-technical teams.
    Experience in working with Agile methodologies and version control systems.
    Proven ability to work independently and manage multiple priorities effectively.

    Education and CertificationBachelors or associate degree in IT disciplineCertification in


    AWS Certified:
    Solution Architect Associate

    or


    AWS Certified:
    Data Engineer

    is added advantage.