Spark & Scala Developer - Pune, India - Radioactive Technologies
Description
Spark & Scala Developer
Exp- 7-13 yrs
Shift- Malaysian / EUR
Full time Contract Opportunity
Fully Remote
please share resume at
Responsibilities-
Create Scala/Spark jobs for data transformation and aggregation
Produce unit tests for Spark transformations and helper methods
Write Scaladoc-style documentation with all code
Design data processing pipelines
Skills-
Scala (with a focus on the functional programming paradigm)
Scalatest, JUnit, Mockito, Embedded Cassandra
Apache Spark 2.x
Apache Spark RDD API
Apache Spark SQL Data Frame API
Apache Spark MLlib API
Apache Spark GraphX API
Apache Spark Streaming API
Spark query tuning and performance optimization
Data Modelling, Data Warehousing, Hadoop, Hive, ETL
SQL database integration ,Microsoft, Oracle, Postgres, and/or MySQL
Experience working with HDFS, S3, Cassandra, and/or DynamoDB
Deep understanding of distributed systems (e.g. CAP theorem, partitioning, replication, consistency, and consensus)