Associate - Data Engineering - Mumbai, India - TIAA

    TIAA
    TIAA Mumbai, India

    2 weeks ago

    tiaa background
    Description

    Specialist - Data Warehousing - INThe role is responsible for gathering, transforming, and storing data through data acquisition, metadata management, data cleansing, data transformation, data distribution and data recovery/backup planning.

    Key Responsibilities and Duties

    They are expected to have specialized knowledge in various ETL (Extract, transform, load) tools such as Informatica, Hyperion etc.
    The applicability of this function is internal as it facilitates leveraging data for decision making purposes.
    Educational Requirements

    University (Degree) PreferredWork Experience

    3+ Years Required; 5+ Years PreferredPhysical Requirements


    Physical Requirements:
    Sedentary Work

    Career Level7IC- Data Pipelines Construction:
    • Design and build efficient, reliable, and scalable data pipelines using Apache Airflow.
    • Develop complex ETL jobs that process and transform large datasets using PySpark and DBT.
    • AWS Glue ETL Development:
    • Design, develop, and maintain scalable ETL jobs using AWS Glue.
    • Utilize AWS Glue Data Catalog for metadata storage and schema versioning.
    • Databrew Recipe Creation:
    • Create and manage Databrew recipes for data preparation and cleansing.
    • Leverage Databrew blueprints to standardize data transformation processes.
    • Data Quality Framework:
    • Architect and implement data quality rules using AWS Glue Data Quality and Databrew.
    • Build automated checks and balances for data validation and anomaly detection.
    • Data Transformation Expertise:
    • Leverage PySpark to perform intricate data manipulations and aggregations at scale.
    • Utilize DBT to manage database transforms declaratively with a focus on modularity and reusability.
    • Testing and Maintenance:
    • Conduct unit tests and develop database validation scripts to ensure data quality and integrity.
    • Troubleshoot and resolve issues in our data pipelines and propose improvements.
    • Process Automation:
    • Automate data processes using Python scripting, focusing on AWS SDK For Pandas for data analysis and manipulation tasks.
    • Implement best practices in version control, testing, deployment, and orchestration of data pipeline workflow.
    • Automate manual processes, optimizing data delivery, re
    • designing infrastructure for greater scalability, etc.
    • AWS

    Analytics Integration:

    • Create and maintain optimal data pipeline architecture for AWS services such as S3, Glue, DataBrew, DQ Rules, etc.
    • Ensure seamless integration of data transformation tools with AWS analytics components.
    • Maintain Data Quality:
    • Establish data quality checks and balance monitoring for all components of pipeline to ensure accurate and available information.
    • Troubleshoot and remediate data processing issues and performance bottlenecks.
    • Collaboration and Support:
    • Work with stakeholders including data, design, product and executive teams to assist with data-related technical issues and support their data infrastructure needs.
    • Document data pipeline processes and contribute to shared knowledge within the team.
    Related SkillsCollaboration, Continuous Improvement Mindset, Data-Driven Business Intelligence, Data Engineering/Analytics, Data Visualization, Predictive Modeling, Problem Solving, Programming, Resourcefulness, Statistics, Story Telling


    _____________________________________________________________________________________________________Company OverviewTIAA Global Capabilities was established in 2016 with a mission to tap into a vast pool of talent, reduce risk by insourcing key platforms and processes, as well as contribute to innovation with a focus on enhancing our technology stack.

    TIAA Global Capabilities is focused on building a scalable and sustainable organization , with a focus on technology , operations and expanding into the shared services business space.

    Working closely with our U.S.

    colleagues and other partners, our goal is to reduce risk, improve the efficiency of our technology and processes and develop innovative ideas to increase throughput and productivity.

    We are an Equal Opportunity/Affirmative Action Employer.

    We consider all qualified applicants for employment regardless of age, race, color, national origin, sex, religion, veteran status, disability, sexual orientation, gender identity, or any other protected status.