Senior Data Engineer - Gurugram, India - Expedia Group

Description

Senior Data Engineer

We are looking for a highly motivated, self-driven , talented and experienced Data Engineer to join our dynamic team and drive the development and maintenance of our data platform. You have at least 9 years of experience as a Data Engineer. An independent thinker, who works around problems and who isn't shy of trying new technologies. Prior experience in working harmoniously with cross-geography team will be an added advantage. You should be equally appropriate in development, test, debugging roles and be ready to wear many hats. This team values "fail-fast" learners and technology enthusiasts who view learning new technology as a fun experience.

What you will do:

• Mentor team across multiple workstreams and lead stakeholder communication across product owners, operations, partner tech teams and business leaders.
• Take architectural ownership for various critical components and systems
• Use experience Designing, developing, deploying and managing high volume/high transactional mission critical SQL/No SQL systems.
• Design, build, and maintain scalable data pipelines and ETL processes Data Warehouse solution.
• Drive proof of concepts and prototypes to accelerate the onboarding of new technologies and solutions to solve business problems
• Develop and optimize data models for efficient storage and retrieval.
• Implement and manage data lakes and warehouses using Spark, Scala, and Hive technologies.
• Demonstrate leadership in mentoring others, establishing standards in documentation and clean code, and setting a professional example to the team
• Demonstrate deep knowledge of optimization techniques and fine tuning SQL queries and stored procedures.
• ETL pipeline orchestration and build using Airflow, Jenkins, Github actions etc..
• Communicate and document solutions and design decisions
• Build bridges between technical teams to enable valuable collaborations
• Evaluate and recommend tools, technologies, and efficient processes
• Learn best practices and how to constantly raise the bar in terms of Engineering Excellence

Who you are:

• 9+ years' experience with managing high transaction volume SQL/No SQL systems like SQL Server, MySQL, PostgreSQL, Teradata
• Ability to lead multiple tasks and projects simultaneously, prioritize decisions and tasks and work in a fast-paced, changing environment
• Proficient in writing complex SQL queries , joins and stored procedures.
• Proven experience as a Data Engineer or similar role.
• Experience with Spark batch and streaming processing and exposure to other bigdata technologies.
• Strong experience in writing spark jobs in Scala and using Hive for data processing and analysis
• Strong understanding of data warehousing concepts and best practices.
• Proficiency in data modelling concepts and can define data elements, relationships, constraints, and rules to ensure data integrity and optimize data storage and retrieval.
• Experience with Apache Iceberg, Delta, and Parquet table formats.
• Excellent problem-solving skills and attention to detail.
• Ability to work independently and collaboratively in a fast-paced environment.
• Demonstrable experience and knowledge in DWH concepts and ETL tools like Informatica
• Strong experience , Strong aptitude for building robust, automated processes and tools
• Amazon AWS experience, particularly RDS, EC2 and Aurora
• Experience in a Linux environment, and familiarity writing shell scripts
• Able to troubleshoot and resolve complex production issues
• Experience with Python or other scripting language
• Experience with Grafana, Splunk, Datadog , CloudWatch or PagerDuty
• Amazon RedShift experience - Nice to have