Data Engineer I Big Data Ecosystems | Fortune 500 Multinational Australian Retailing Firm | 3-6 years (BB-E8CDA)

Found in: Neuvoo Premium IN


Key goals for the position:
• Develop technical solutions for a data platform based on existing and emerging patterns and technologies
• Enable advanced analytics including AI/ML and data-driven decision making by building pipelines that are event driven, can transform/ cleanse/ manipulate data and facilitate visualization of data integrated from enterprise systems, internet, and many other disparate sources
• Enable data exchange amongst enterprise systems via real time event streaming platform
• Champion conceptual and technical aspects of real time data platform development and support, using various design patterns, dynamically scalable technologies, and Devops practices
Team Member Minimum Requirements
Preferred Formal Education/Qualifications:
• Bachelor of Computer Science
• Other qualifications in IT or related discipline Proven Experience:
• Developing, Testing and Supporting data lakes, data warehouses and large-scale data processing systems.
• Working effectively in data platform scrum teams for technical design, development and deployment of solutions.
• Implementing and supporting data platform technologies both on-premise and cloud-based.
• Deep understanding and adoption of Agile delivery techniques, including Continuous Integration & Continuous Delivery (CI/CD).
• Developing solutions using a variety of technologies and tools to marry on-premise and cloud-based systems together.
• Participate in technology/tools/framework evaluation to recommend/influence adoption
• Follow best practices to ensure high standards of data availability, reliability, completeness, efficiency and quality. Skills/Knowledge/Abilities/Technology Used:
• Advanced SQL working knowledge and experience working with a variety of relational databases, SQL query authoring.
• Expert understanding and hands-on working knowledge of message queueing, real time event streaming architecture for data platforms including Kafka, Kinesis, SNS, SQS etc.
• Deep understanding and ability to build automations for data transformation, processing of data structures, metadata, dependencies and workload management for very high volume, velocity and variety of data.
• Deep working knowledge of AWS technologies such as S3, EC2, EMR or Bigdata/Hadoop, RDS, Lambda, Elasticsearch, Redshift, Cloudformation, Terraform, Cloudwatch etc.
• Sound to advanced level working knowledge of Git and CI/CD pipeline technologies such as Jenkins, Chef, Kubernetes & Docker containers
• Deep experience with object-oriented/object function scripting languages: Python, Scala, Java, R, C++, Golang etc.
• Working knowledge of Data warehouses, NoSQL databases and ETL technologies like Informatica, Talend etc
• Understanding of modern SaaS based monitoring and logging tools like New Relic/Sumologic or equivalent
• Must have experience with Linux, shell scripting
• Familiarity with BI tools a plus
Area of Accountability Key Responsibilities & Deliverables Performance Measures & Targets
Development and Operations:
• Collaborate with team members to design and implement data solutions in alignment with the project schedule.
• Code, test, and document new or modified data systems to create robust and scalable applications for data analytics.
• Peer and customer feedback
• Line manager observations
• Line manager observations
• Create data flow diagrams for business systems.
• Implement security as part and parcel of all development.
• Builds automation tools to provide a self-service data platform and enable CICD
• Creates and maintains a data catalogue.
• Develops standards and processes.
• Champions change management
• Take Care Safety • Accountable for a safe site for everyone, every day by implementing and evaluating safe work practices, improving safety performance and
celebrating safety achievements
• Follows a Devops mindset and practice
• Conducts product demonstrations, showcases, briefings on trending technologies, processes and solutions relating to the data platform.
• Line manager observations
• Line manager observations
• Communicates effectively with stakeholders, partners, vendors.
• Peer and customer feedback

Other Areas of Accountability Key Responsibilities/Major Activities:
Values and Behaviour
• Live Integrity
• Think Customer
• Grow Together
• Reach Higher

calendar_today4 days ago

Similar jobs

location_onNelamangala, India


I expressly authorise the Terms and Conditions