Data Streaming - Bengaluru, India - ACE Talent Consulting

ACE Talent Consulting
ACE Talent Consulting
Verified Company
Bengaluru, India

1 month ago

Deepika Kaur

Posted by:

Deepika Kaur

beBee Recuiter
Description

ob Description:
Spark/Scala/PySpark developer who knows how to fully exploit the potential of our Spark cluster.


Should have ability to clean, transform, and analyze vast amounts of raw data from various systems using Spark to provide ready-to-use data.


Responsibilities:

Create Scala/Spark/Pyspark jobs for data transformation and aggregation

Produce unit tests for Spark transformations and helper methods

Write Scaladoc-style documentation with all code

Design data processing pipelines


Skills:
Pyspark

Scala (with a focus on the functional programming paradigm)

Apache Spark 2.x, 3.x

  • Apache Spark RDD API
  • Apache Spark SQL DataFrame API
  • Apache Spark Streaming API
Spark query tuning and performance optimization

SQL database integration (Postgres, and/or MySQL)

Experience working with HDFS, AWS ( S3, Redshift, EMR, IAM, Polices, Routing)

CI-CD Pipleline, Jenkins, Gitlab /Bitbucket

Deep understanding of distributed systems (e.g. partitioning, replication, consistency, and consensus)


Salary:
₹300, ₹1,500,000.00 per year


Schedule:

  • Day shift
  • Monday to Friday

Ability to commute/relocate:

  • Bengaluru, Karnataka: Reliably commute or planning to relocate before starting work (required)

Experience:

- total work: 3 years (preferred)


Work Location:
One location
More jobs from ACE Talent Consulting
See all jobs of ACE Talent Consulting