Data Engineering Quality Assurance Engineer - Bengaluru, India - CoffeeBeans

Description

We are seeking a Data Engineering Quality Assurance Engineer to ensure the integrity and reliability of our large-scale data engineering projects. The ideal candidate will have a solid foundation in data engineering concepts and experience in automated testing frameworks. You will play a critical role in maintaining the quality of our data pipelines and infrastructure, ensuring they meet the highest standards of accuracy and performance.

What will you do?

Develop and implement automated testing frameworks to validate the performance and reliability of data pipelines and architectures.

Work closely with data engineers to understand data workflows, schemas, and processing logic to create effective test cases.

Identify, document, and track bugs and issues within data processes, collaborating with engineering teams to facilitate fixes.

Perform both functional and non-functional testing, including performance, scalability, and stress testing.

Monitor data processes and outputs for anomalies and deviations from expected patterns, initiating corrective actions without delay.

Maintain documentation for QA processes and metrics, ensuring adherence to quality standards and compliance with regulations.

Continuously update and refine testing strategies to incorporate new technologies and methodologies.

What are we looking for?

2-4 years of experience in data engineering or quality assurance roles with a focus on data-heavy projects.

Strong proficiency in SQL and scripting languages such as Python or Bash for automation.

Experience with automated testing tools and frameworks (e.g., Selenium, TestComplete, JMeter).

Knowledge of data modeling, data warehousing, and ETL processes.

Familiarity with big data technologies (e.g., Hadoop, Spark, Kafka) and cloud services (e.g., AWS, Azure, Google Cloud).

Proven ability to work in an Agile development environment and adapt to a startup-like culture, fostering innovation and rapid growth.Desirable Skills:

Experience with continuous integration/continuous deployment (CI/CD) practices and tools.

Knowledge of advanced analytics and machine learning concepts to understand the data lifecycle fully.

Certification in quality assurance or relevant fields.