Big Data Specialist - Bengaluru, India - o9 Solutions, Inc.
Description
Be part of something revolutionary.At o9 Solutions, our mission is clear: be the Most Valuable Platform (MVP) for enterprises.
With our AI-driven platform — the o9 Digital Brain — we integrate global enterprises' siloed planning capabilities, helping them capture millions and, in some cases, billions of dollars in value leakage.
But our impact doesn't stop there. Businesses that plan better and faster also reduce waste, which drives better outcomes for the planet, too.We're on the lookout for the brightest, most committed individuals to join us on our mission.
Along the journey, we'll provide you with a nurturing environment where you can be part of something truly extraordinary and make a real difference for companies and the planet .
About the role....Design and build data pipelines with an emphasis on scale, performance, and reliability.
Provide technical expertise in the areas of design and implementation of planning solutions powered by Delta Lake Lakehouses implemented in-house.
Partner with the data teams, and enterprise architecture organization to ensure the best use of standards for the key data domains and use cases.
Continuous learner with an eye on emerging trends around data lake architecture and enterprise data solutions.Support retail planning implementations which are time-critical
What you will do.......
Distributed Compute and StorageWrite PySpark code, debug Spark errors, perform Root Cause Analysis and suggest solutions/workarounds.
Familiarity with Spark settings and changes needed to reduce run time.
Modify data storage and read/write architecture of existing distributed workloads to optimize the runtime.
Work with big data constructs such as Spark, HDFS, HIVE, and DeltaLake and apply them to o9 needs.
Work with containers and K8, Cloud constructs like S3 and large VM clusters to support Retail Supply Planning workloads.
Mentoring and Training
Package, document and create training content with active demonstrations and iterative workshops with the broader implementation and customer teams. Educate cross-functional teams during the product life cycle.
Coach and ramp up team members to become independent contributors.
ProductParticipate in the process of collecting and understanding business / technical requirements with external (customers) and internal teams.
Work with Architects to translate business requirements to functional specifications and use cases and work with them to implement such features in the Supply Planning Product.
Help the team to improve platform capabilities.What you will have....
Must Have
BE, MCA or MS degree in Computer Science or Information Technology5-8 years of experience building solutions in big data technologies.
Strong experience programming with both Python, Pyspark and SPARK SQL.Hands-on experience designing and building streaming data pipelines using Airflow, Kafka etc.
Must have experience building Data Lake and data warehouse solutions using ETL, and ELT pipelines on any of Databricks, Snowflake, Azure Data Lake etc.
Strong understanding of database and analytical technologies in the industry including modelling with OLAP databasesStrong understanding of cloud platforms like AWS, Azure, or GCP, and their services (e.g., EC2, S3, AKS, EKS, etc.).Experience in continuous delivery through containers, and orchestration technologies.
Experience with Data Manipulation Libraries like Ray, and Polars is mandatory.Ability to prioritize and manage work to critical project timelines in a fast-paced environment.
Nice to have :
Experience with graph manipulation libraries on Spark (like GraphX) is an added advantage.
Supply Chain Planning software industry experience is an added advantage.
Databricks Certified Associate Developer for Apache Spark 3.0Added advantage for experience in Scala on Spark.
Added advantage for experience with HDFS and HQLAWS or any public cloud certification is a must.
More about us...
With the latest increase in our valuation from $2.7B to $3.7B despite challenging global macroeconomic conditions, o9 Solutions is one of the fastest-growing technology companies in the world today.
Our platform, the o9 Digital Brain, is the premier AI-powered, cloud-native platform driving the digital transformations of major global enterprises including Google, Walmart, ABInBev, Starbucks and many others.
Our headquarters are located in Dallas, with offices in Amsterdam, Paris, London, Barcelona, Madrid, Sao Paolo, Bengaluru, Tokyo, Seoul, Milan, Stockholm, Sydney, Shanghai, Singapore and Munich.