Lead Data Engineer - Secunderabad, India - Talent500

    Talent500
    Talent500 Secunderabad, India

    Found in: Talent IN 2A C2 - 1 week ago

    Talent500 background
    Description

    Position Summary:

    Land O'Lakes is looking to add a Senior Data Engineer to our Winfield United Data & Analytics product team. The Senior data engineer will design & execute solutions and work with a team of engineers to ensure consistent, reliable, efficient and sustainable solutions are delivered. The data engineer will also contribute towards improving data management processes that result in highly accurate and trusted data assets for use in Winnfield United and across the organization.

    Job Description & Responsibilities:

    • Ability to obtain a clear understanding of business needs and value, mapping out the solution, and working on its implementation.
    • Ability to collaborate across all levels and areas of the product & business team as well as perform hands-on implementation.
    • Excellent communication, presentation, and documentation skills, including the ability to distill highly technical concepts and communicate them effectively to non-technical stakeholders
    • Capable of using agile methodology and implementing Continuous Integration / Continuous Delivery (CI/CD) pipelines
    • Exposure to data vault method, model, and architecture
    • Ability to monitor the overall data landscape: meta data, processing patterns, and data quality
    • Ability to work closely with developers and data analysts to design and optimize data access, query, reporting and analysis strategies to solve complex business problems
    • Ability to proactively communicate and manage technology and projects risks
    • Troubleshooting skills, ability to determine impacts, ability to resolve complex issues, and ability to exercise sound judgment and initiative in challenging situations
    • Establish platform-wide standards
    • Perform root cause analysis when there are issues
    • Ability to proactively communicate and manage technology and projects risks
    • Troubleshooting skills, ability to determine impacts, ability to resolve complex issues, and ability to exercise sound judgment and initiative in challenging situations.

    Experience Required:

    • Strong experience in working with heterogeneous datasets in building and optimizing data pipelines, pipeline architectures and integrated datasets using various data integration technologies (ETL/ELT, data replication/CDC, message-oriented data movement, API design, etc.)
    • Hands on experience with programming languages such as Python, SQL to create automated processes for data gathering, cleansing, reporting, and visualization

    Experience Desired:

    • 3+ years of experience leading engineering resources
    • Experience with DevOps, CI/CD pipelines and Databricks
    • Experience working with Qlik technologies (Replicate and Compose)
    • Experience with any scripting languages, preferably Python
    • Experience working using agile methodologies
    • Experience in Manufacturing, Retail or Agriculture industry

    Education and Training Required:

    • Bachelor's or Master's degree in Business Analytics, Computer Science, Mathematics, Statistics, or a related discipline; or an equivalent combination of demonstrated skills and experience

    Primary Skills:

    • 7+ years of experience with SQL & data engineering techniques.
    • 7+ years of experience building data integration solutions using tools like Informatica, Talend, Mulesoft, Qlik, etc.
    • Experience in working with Cloud data warehouse like Snowflake, AWS Redshift, Azure Synapse or Google BigQuery.

    Additional Skills:

    • Collaborate with cross functional team to resolve data quality and operational issues.
    • Demonstrating best practices for code development and optimization, unit testing, CI/CD, performance testing, capacity planning, documentation, monitoring, alerting, and incident response in order to ensure data availability, data quality, usability and required performance.
    • Ability to work closely with business partners to influence decisions on technology investments
    • Ability to provide technical leadership to evangelize best practices / proven patterns and promote the use of Corporate IT standards
    • Working knowledge of data privacy regulations including GDPR and CCPA as well as how to design solutions to drive compliance