Spark & Scala Developer - Pune, India - Radioactive Technologies

    Radioactive Technologies
    Radioactive Technologies Pune, India

    1 week ago

    Default job background
    Technology / Internet
    Description

    Spark & Scala Developer

    Exp- 7-13 yrs

    Shift- Malaysian / EUR

    Full time Contract Opportunity

    Fully Remote

    please share resume at

    Responsibilities-

    Create Scala/Spark jobs for data transformation and aggregation

    Produce unit tests for Spark transformations and helper methods

    Write Scaladoc-style documentation with all code

    Design data processing pipelines

    Skills-

    Scala (with a focus on the functional programming paradigm)

    Scalatest, JUnit, Mockito, Embedded Cassandra

    Apache Spark 2.x

    Apache Spark RDD API

    Apache Spark SQL Data Frame API

    Apache Spark MLlib API

    Apache Spark GraphX API

    Apache Spark Streaming API

    Spark query tuning and performance optimization

    Data Modelling, Data Warehousing, Hadoop, Hive, ETL

    SQL database integration ,Microsoft, Oracle, Postgres, and/or MySQL

    Experience working with HDFS, S3, Cassandra, and/or DynamoDB

    Deep understanding of distributed systems (e.g. CAP theorem, partitioning, replication, consistency, and consensus)