Principal Data Engineer - Pune, India - NorthStar HR Consultants

Description

Job Title - Principal Data EngineerJob Location - Baner, Pune

About Company & ProductOur client is a pioneering Identity SecOps provider that arms identity and cybersecurity teams with end-to-end, real-time identity security posture management and threat detection.

They are designed to secure identity and access in any cloud (e.g. AWS, Azure), as well as on-premises or data centres. In 2022, Gartner recognized our client as a Cool Vendor in Identity-First Security.
The product is heavily driven by data analytics as well as AI/ML (in the near future).

The core idea is to analyze events & logs from enterprise systems & networks to enable the use cases we are trying to solve in the area of identity & access security.

Position OverviewAs a Principal Data Engineer, you will have end-to-end ownership of the relevant software components, ensuring scalability, reliability, and optimal performance.

You will actively contribute to the research, design and development of enterprise-grade, scalable data solutions that enable our organisation to extract maximum value from data assets.

Collaboration with cross-functional teams, including QA, UI, DevOps, IT/Ops, and non-engineering stakeholders, is integral to this role.

ResponsibilitiesResearch, design, and develop components for our enterprise-level cybersecurity application.
Design, develop, and deploy scalable and efficient data pipelines, ETL processes, and data integration solutions.
Architect and implement data models, schemas, and database solutions to support our analytical and operational needs.
Evaluate and select appropriate technologies, tools, and frameworks to enhance data processing, storage, and analytics capabilities.
Drive initiatives to improve data quality, consistency, and reliability across the organization.
Lead and contribute to the testing, validation and documentation of data solutions to ensure scalability, reliability, and maintainability.
Identify areas of improvement, enhancements, and innovation in the software development process.
Collaborate closely with the engineering team and non-engineering stakeholders throughout the development lifecycle.
Thrive in a dynamic startup environment, contributing to a culture of innovation and excellence.
Write clean, maintainable, and efficient code, adhering to best practices and coding standards.
Participate in code reviews and provide constructive feedback to peers.
Collaborate with the DevOps team to deploy and manage applications in production environments.
Stay up-to-date with industry trends and advancements to ensure our technology stack remains current and competitive.

RequirementsAt least 14 years of hands-on experience in designing and developing enterprise-grade software, including 6+ years of experience in data engineering.

Proven track record of designing and implementing large-scale data solutions.
Strong hands-on experience with big data technologies, frameworks and distributed computing.
Excellent problem-solving abilities, analytical thinking, and attention to detail.
Solid proficiency in one or more programming languages such as Python, Java, or Ruby.

Familiarity with cloud platforms (e.g., AWS, Azure, or Google Cloud) and containerization technologies (e.g., Docker, Kubernetes).Knowledge of RESTful API design and implementation.

Experience with the ELK stack and NoSQL databases (e.g., MongoDB) is a significant advantage.
Strong communication and collaboration skills.
Ability to thrive in a fast-paced, dynamic work environment.
Master's or Bachelor's degree in Computer Science, Engineering, or a related field.