AWS Data Engineer - Hyderabad, India - Maverick Edutech Private Limited

Description

JobDescription:

Asan AWS Data Engineer you will be responsible for designingimplementing and managing data pipelines and architectures onAmazon Web Services (AWS) to support our datadriven initiatives.You will work closely with crossfunctional teams including datascientists analysts and software engineers to deliver scalable andreliable solutions that enable us to derive insights and value fromour dataassets.

Responsibilities:

• Designand implement scalable and reliable data architectures on AWSleveraging services such as Amazon S3 Amazon Redshift Amazon EMRand AWSGlue.
  
• Developand maintain data pipelines for ingesting processing andtransforming data from various sources intoAWS.
  
• Implementdata governance and quality control measures to ensure theintegrity and consistency of ourdata.
  
• Collaboratewith data scientists and analysts to provide them with access toclean and wellorganized data for analysis andmodeling.
  
• Optimizedata pipelines and infrastructure for performance costeffectivenessandscalability.
  
• Ensurecompliance with data security and privacy regulations implementingencryption access controls and monitoring mechanisms asneeded.
  
• Documentdesigns processes and best practices for knowledge sharing andfuturereference.
  
• Stayupdated on the latest AWS services and technologies and recommendinnovative solutions to enhance our datacapabilities.
  

AWSSpark and SparkStreaming

AWSRedshift

AWSS3

AWSDMS for Data Replication Nice toHave

AWSStep FunctionsAirflow

AWSGlue ispreferred

Goodunderstanding of AWS Data and Analyticssolutions.

Ableto understand the clients requirements and project requirements anddevelop low latency (near real time and batch) data integrationsolutions.