Lead Data Engineer - Kerala - Experion Technologies

    Experion Technologies
    Experion Technologies Kerala

    1 day ago

    ₹3,000,000 - ₹6,500,000 (INR) per year * Technology / Internet
    Description

    Key Responsibilities


    • Data Ingestion Framework:

    o Design & Development: Architect, develop, and maintain an end-to-end data ingestion framework that efficiently extracts, transforms, and loads data from diverse sources.

    o Framework Optimization: Use AWS services such as AWS Glue, Lambda, EMR, ECS , EC2 and Step Functions to build highly scalable, resilient, and automated data pipelines.


    • Data Quality & Validation:

    o Validation Processes: Develop and implement automated data quality checks, validation routines, and error-handling mechanisms to ensure the accuracy and integrity of incoming data.

    o Monitoring & Reporting: Establish comprehensive monitoring, logging, and alerting systems to proactively identify and resolve data quality issues.


    • API Development:

    o Design & Implementation: Architect and develop secure, high-performance APIs to enable seamless integration of data services with external applications and internal systems.

    o Documentation & Best Practices: Create thorough API documentation and establish standards for API security, versioning, and performance optimization.


    • Collaboration & Agile Practices:

    o Cross-Functional Communication: Work closely with business stakeholders, data scientists, and operations teams to understand requirements and translate them into technical solutions.

    o Agile Development: Participate in sprint planning, code reviews, and agile ceremonies, while contributing to continuous improvement initiatives and CI/CD pipeline development (using tools like GitLab).

    Required Qualifications


    • Experience & Technical Skills:

    o Professional Background: At least 5 years of relevant experience in data engineering with a strong emphasis on analytical platform development.

    o Programming Skills: Proficiency in Python and/or PySpark, SQL for developing ETL processes and handling large-scale data manipulation.

    o AWS Expertise: Extensive experience using AWS services including AWS Glue, Lambda, Step Functions, and S3 to build and manage data ingestion frameworks.

    o Data Platforms: Familiarity with big data systems (e.g., AWS EMR, Apache Spark, Apache Iceberg) and databases like DynamoDB, Aurora, Postgres, or Redshift.

    o API Development: Proven experience in designing and implementing RESTful APIs and integrating them with external and internal systems.

    o CI/CD & Agile: Hands-on experience with CI/CD pipelines (preferably with GitLab) and Agile development methodologies.


    • Soft Skills:

    o Strong problem-solving abilities and attention to detail.

    o Excellent communication and interpersonal skills with the ability to work independently and collaboratively.

    o Capacity to quickly learn and adapt to new technologies and evolving business requirements.

    Preferred Qualifications


    • Bachelor's or Master's degree in Computer Science, Data Engineering, or a related field.


    • Experience with additional AWS services such as Kinesis, Firehose, and SQS.


    • Familiarity with data lakehouse architectures and modern data quality frameworks.


    • Prior experience in a role that required proactive data quality management and API- driven integrations in complex, multi-cluster environments.

    * This salary range is an estimation made by beBee
  • Work in company

    Data Engineer

    Only for registered members

    Client looking for a skilled Data Engineer with strong experience in Python, PySpark, and cloud-based data engineering tools to support the development of scalable ETL pipelines and data processing systems. · ...

    Kerala

    1 week ago

  • Work in company

    Data Engineer

    Only for registered members

    We are looking for a skilled Data Engineer with strong experience in Python, PySpark, · AWS S3 and AWS Glue to support the development of scalable ETL pipelines · and data processing systems. · Build and maintain ETL pipelines for processing structured · and semi-structured data. ...

    Kerala

    1 week ago

  • Work in company Remote job

    Senior Data Engineer

    Only for registered members

    We are an AI and Data Consulting Startup transforming how businesses leverage technology through four core service lines: · We're a small, fast-moving team delivering enterprise-grade solutions with startup agility. · You will design, build, and optimize data lakehouse solutions, ...

    Kerala

    1 month ago

  • Work in company Remote job

    Senior Data Engineer

    Only for registered members

    Key Note: Applicants must be willing to work in EST Hours. · Responsibilities · Design, implement, and manage complex DAGs and SubDAGs in Apache Airflow. · Utilize Operators, macros, connections, variables, and xcom for efficient task execution. · Conduct unit testing to ensure t ...

    Kerala ₹2,200,000 - ₹4,500,000 (INR) per year

    2 days ago

  • Work in company

    Chisquare Labs

    Only for registered members

    We are looking for a Data Engineer with a strong focus on data migration testing and data quality to help validate the movement of data from several PostgreSQL and other databases into our centralized data warehouse. · ...

    Kerala

    3 weeks ago

  • Work in company

    Data Visualization Lead

    Only for registered members

    We are looking for an experienced Data Visualization Lead who can transform complex sensor data into reliable insights to support our core activities. · This role requires leading development of data analysis solutions, designing robust pipelines, defining visualization standards ...

    Kerala

    4 weeks ago

  • Work in company

    Junior Software Engineer

    Only for registered members

    This is a full-time on-site role for a Junior Software Engineer - Data Pipeline, based in Kerala, India. The role involves designing, developing and maintaining efficient and scalable data pipelines for the organization's projects. Daily tasks include data extraction transformati ...

    Kerala

    1 month ago

  • Work in company Remote job

    Tableau Cloud Administrator – Data Foundations

    Only for registered members

    As a Tableau Cloud Administrator you will help lead the evolution of self-serve analytics at Riot by ensuring our Tableau environment is well-governed optimized and designed for high performance and usability. · Administer and optimize the Tableau Cloud environment including site ...

    Kerala

    1 week ago

  • Work in company

    Senior Data Scientist

    Only for registered members

    We are seeking an experienced Senior Data Scientist to lead the design and development of advanced data models, machine learning solutions, and analytics platforms that power our AI products and business solutions. · ...

    Kerala

    1 month ago

  • Work in company

    DPS Engineer

    Only for registered members

    We are looking for a dedicated and experienced DPS Engineer to join our team in Alkhobar, KSA. In this role, you will be responsible for the deployment, maintenance, and optimization of our data protection solutions. Your expertise in data protection strategies aWe are seeking a ...

    KErala

    1 day ago

  • Work in company

    Computer Vision Engineer

    Only for registered members

    Model development and deployment involve designing computer vision models for tasks such as detection and classification. · Degree in Computer Science or related field. · ...

    Kerala

    3 weeks ago

  • Work in company

    SQL & SSIS Engineer

    Only for registered members

    We are seeking a skilled SQL & SSIS Engineer with hands-on experience in MS SQL Server and SSIS to maintain and support enterprise data integration solutions. · Candidates with a SQL Developer background and strong SSIS exposure are also welcome if they are comfortable working in ...

    Kerala

    1 month ago

  • Work in company

    Software Engineer

    Only for registered members

    One of Our Client is looking for a React Native Engineer with 3 to 7 years of experience. The ideal candidate develops UIs using React Native and React, and builds reusable, and scalable components. They are proficient in developing robust APIs with Node.js, integrating various A ...

    Kerala

    2 weeks ago

  • Work in company

    AI Architect

    Only for registered members

    We are seeking a highly skilled and forward-thinking AI Architect to design and lead the architecture of our AI platforms and solutions. · Lead the end-to-end process of developing and delivering a Gen AI-based solution that provides actionable insights from business data and sup ...

    Kerala

    1 month ago

  • Work in company

    Firmware Engineer intern

    Only for registered members

    Company Description · Hashco Solutions combines technical expertise with a focus on industry trends to create innovative solutions that meet current demands while anticipating future needs. We specialize in delivering scalable, reliable, and tailored IT solutions to help business ...

    Kerala

    2 days ago

  • Work in company Remote job

    Search Engine Optimization Manager

    Only for registered members

    Company Description · We Buy Dead Stocks, based in Dubai, specializes in helping businesses efficiently convert excess inventory into working capital through fast and fair solutions. Trusted by importers, wholesalers, and retailers across the UAE, GCC, and globally, our expertise ...

    Kerala

    1 day ago

  • Work in company

    Senior Software Engineer

    Only for registered members

    About the job : · Senior Software Engineer - SQL · ...

    Kerala

    3 weeks ago

  • Work in company

    Information Technology Support Specialist

    Only for registered members

    We're looking for a Cloud Security & Compliance Engineer to own the setup, hardening, and ongoing security of our Microsoft 365 and Azure environments. · -Design and implement secure Microsoft 365 tenant configuration from the ground up · -Configure and manage Azure infrastructur ...

    Kerala

    1 month ago

  • Work in company

    Junior Full Stack Engineer

    Only for registered members

    We are looking for a motivated Full Stack Engineer with AI API integration skills to support mission-critical applications. · Develop, test, and deploy full stack applications with AI/ML features. · Implement and support AI API integrations across Claude, OpenAI, Gemini (Google), ...

    Kerala

    3 weeks ago

  • Work in company

    AI Product Manager

    Only for registered members

    We are seeking an experienced and driven AI Product Manager to lead the strategy, design, and delivery of our AI products. · Perform Market research to understand the problem clearly. · Design product strategy and roadmap so as to serve different clients. · ...

    Kerala

    1 month ago

Jobs
>
Lead data engineer