Jobs
>
Chennai

    Senior Staff Software Engineer, Data PlatformChennai, India - chennai, India - Udemy

    Udemy
    Udemy chennai, India

    2 weeks ago

    Default job background
    Description

    About this role

    Are you excited about building a Data Mesh platform on a global scale? Do you want to build a self-service, highly scalable platform to power Udemy's groundbreaking AI, ML, and data products? Udemy's Data org is looking for a self-driven, creative Data Mesh platform engineer passionate about changing the world by democratizing online education.

    We are building highly available, scalable data infrastructure, services, tools, and libraries to serve critical data discovery, ML/data pipeline development & deployment, data security & compliance, and self-healing data operation needs.

    What you'll be doing

  • Design, build, enhance, and support self-service data infrastructure as a platform, which would be used for Udemy's data mesh.
  • Build robust, scalable, cloud-native, cost-optimized real-time and batch data integration pipelines in AWS with Airflow, Kafka, Databricks, Delta Lake, AWS services like S3, EMR, Athena, RDS, Redshift, EKS, etc., and Snowflake.
  • Contribute towards Udemy's comprehensive data quality initiatives like data contracts, data validation, observability, and alerting
  • Support data platform privacy with RBAC and encryption
  • On-call rotation business hours

    • What you'll have

  • 8+ years of experience in building distributed systems with Java, Scala, or Python
  • 5+ years of experience managing big data platforms with storage (AWS S3), compute (AWS EMR or Databricks with Spark, AWS Athena or Presto), data warehouses (Redshift or Snowflake), streaming (Kafka or AWS Kinesis)
  • 5+ years of experience in agile development methodologies & test-driven development (TDD)
  • 5+ years of experience building and maintaining resilient real-time & batch data pipelines.
  • Experience in building or using any one of the data quality tools (in-house built, SODA SQL, Anomalo, Monte Carlo, Great Expectations, or any other)
  • Expertise in cloud data privacy tools in the area of authentication (AWS IAM/OKTA), encryption (AWS KMS/Safenet), authorization (Apache Ranger/Privacera)
  • Strong grasp of object-oriented design patterns, algorithms, and data structures

    • We understand that not everyone will match each of the above qualifications. However, we also realize that everyone has unique experiences that can add value to our company. Even if you think your background might not perfectly align, we'd love to hear from you

  • Udemy

    Senior Staff Software Engineer, Data PlatformChennai, India

    1 week ago

    Udemy Chennai, India

    About this role · Are you excited about building a Data Mesh platform on a global scale? Do you want to build a self-service, highly scalable platform to power Udemy's groundbreaking AI, ML, and data products? Udemy's Data org is looking for a self-driven, creative Data Mesh plat ...