No more applications are being accepted for this job
- Mentor team across multiple workstreams and lead stakeholder communication across product owners, operations, partner tech teams and business leaders.
- Take architectural ownership for various critical components and systems
- Use experience Designing, developing, deploying and managing high volume/high transactional mission critical SQL/No SQL systems.
- Design, build, and maintain scalable data pipelines and ETL processes Data Warehouse solution.
- Drive proof of concepts and prototypes to accelerate the onboarding of new technologies and solutions to solve business problems
- Develop and optimize data models for efficient storage and retrieval.
- Implement and manage data lakes and warehouses using Spark, Scala, and Hive technologies.
- Demonstrate leadership in mentoring others, establishing standards in documentation and clean code, and setting a professional example to the team
- Demonstrate deep knowledge of optimization techniques and fine tuning SQL queries and stored procedures.
- ETL pipeline orchestration and build using Airflow, Jenkins, Github actions etc..
- Communicate and document solutions and design decisions
- Build bridges between technical teams to enable valuable collaborations
- Evaluate and recommend tools, technologies, and efficient processes
- Learn best practices and how to constantly raise the bar in terms of Engineering Excellence
- 9+ years' experience with managing high transaction volume SQL/No SQL systems like SQL Server, MySQL, PostgreSQL, Teradata
- Ability to lead multiple tasks and projects simultaneously, prioritize decisions and tasks and work in a fast-paced, changing environment
- Proficient in writing complex SQL queries , joins and stored procedures.
- Proven experience as a Data Engineer or similar role.
- Experience with Spark batch and streaming processing and exposure to other bigdata technologies.
- Strong experience in writing spark jobs in Scala and using Hive for data processing and analysis
- Strong understanding of data warehousing concepts and best practices.
- Proficiency in data modelling concepts and can define data elements, relationships, constraints, and rules to ensure data integrity and optimize data storage and retrieval.
- Experience with Apache Iceberg, Delta, and Parquet table formats.
- Excellent problem-solving skills and attention to detail.
- Ability to work independently and collaboratively in a fast-paced environment.
- Demonstrable experience and knowledge in DWH concepts and ETL tools like Informatica
- Strong experience , Strong aptitude for building robust, automated processes and tools
- Amazon AWS experience, particularly RDS, EC2 and Aurora
- Experience in a Linux environment, and familiarity writing shell scripts
- Able to troubleshoot and resolve complex production issues
- Experience with Python or other scripting language
- Experience with Grafana, Splunk, Datadog , CloudWatch or PagerDuty
- Amazon RedShift experience - Nice to have
Senior Data Engineer - Gurugram, India - Expedia Group
Description
Senior Data Engineer
We are looking for a highly motivated, self-driven , talented and experienced Data Engineer to join our dynamic team and drive the development and maintenance of our data platform. You have at least 9 years of experience as a Data Engineer. An independent thinker, who works around problems and who isn't shy of trying new technologies. Prior experience in working harmoniously with cross-geography team will be an added advantage. You should be equally appropriate in development, test, debugging roles and be ready to wear many hats. This team values "fail-fast" learners and technology enthusiasts who view learning new technology as a fun experience.
What you will do:
Who you are: