Jobs
>
Hyderabad

    Bigdata Scala Spark Data Engineer - Hyderabad, India - LTIMindtree

    LTIMindtree
    LTIMindtree Hyderabad, India

    Found in: Appcast Linkedin IN C2 - 7 hours ago

    LTIMindtree background
    Technology / Internet
    Description

    Primary Skill – Scala Spark

    Total Exp – 5.2 Years to 13 Years

    Notice Period – ONLY 0 to 60 Days Joiners

    Job Location – Pune, Mumbai, Chennai, Hyderabad, Bangalore, Kolkata, Coimbatore, Delhi

    Job Description:

    Job Description:

    JD: Spark + scala Tech Lead

    Job Description

    We are looking for a Spark Scala Tech Lead with 5+ Years experience who knows how to fully exploit the potential of Spark cluster. You will be responsible to clean, transform, and analyze vast amounts of raw data from various systems using Spark to provide ready-to-use data to feature developers and business analysts.

    This involves both ad-hoc requests as well as data pipelines that are embedded in production environment.

    Responsibilities


    • Create Scala/Spark jobs for data transformation and aggregation


    • Produce unit tests for Spark transformations and helper methods


    • Write Scaladoc-style documentation with all code


    • Design data processing pipelines

    What you'll be doing:


    • Building distributed and highly parallelized Big data processing pipeline which process massive amount of data (both structured and unstructured data) in near real-time


    • Leveraging Spark to enrich and transform corporate data to enable searching, data visualization, and advanced analytics


    • Working closely with analysts and business stakeholders to develop analytics models


    • Continuous delivering on Hadoop and other Big Data Platforms


    • Automating?processes where possible and are repeatable and reliable


    • Working closely with QA team

    As an expert Data Engineer, you are expected to have:


    • Excellent Programming language skills: Scala, Python, Bash, SQL, Bash


    • Excellent understanding of Hadoop and HDFS.


    • Excellent understanding of file types, their pros and cons


    • Excellent understanding of Build Tools especially SBT/Gradle


    • Very good understanding and good experience in implementing CICD e.g. Building and Maintaining Jenkins Pipelines


    • Working experience in SAFe/Agile


    • As an expert, you are also responsible for Juniors writing maintainable code


    • Are collaborative and achieve your expectations through communication and teamwork


    • Are curious, responsive and can understand the needs of others to ensure delivery of the desired results


    • Work qualitatively and strive to always do things better

    Skills


    • Spark + Scala Scala (with a focus on the programming )


    • Apache Spark 2.x


    • Spark query tuning and performance optimization


    • Understanding on Hadoop Architecture


    • Experience working with Hive, SQL and HDFS


    • Deep understanding of distributed systems (e.g. CAP theorem, partitioning, replication, consistency, and consensus)


    • Good to have experience in Scala


    • Experience working with tools like Jenkins, Jira, Bitbucket and Git


    • Experience in writing shell scripts and working with Linux platforms.

    As an advantage, familiarity with one or more of the below technologies:


    • Experience?working in Agile/SAFe


    • Experience/certification with AWS Cloud

  • Anblicks

    Data Engineer

    Found in: Talent IN 2A C2 - 6 days ago

    Anblicks Hyderabad, India

    Designation: Data Engineer · Skills: Python, Pyspark and AWS · Experience: 5+ years · Responsibilities: · Design, develop, and maintain scalable data pipelines and ETL processes · Work closely with data scientists and analysts to understand business requirements and translate t ...

  • Infomatics Corp

    Data Engineer

    Found in: Talent IN 2A C2 - 6 days ago

    Infomatics Corp Hyderabad, India

    Position Overview: We are seeking a highly skilled and motivated Data Engineer to join our dynamic team. As a Data Engineer, you will play a critical role in designing, building, and maintaining our data infrastructure and systems. You will collaborate closely with cross-function ...

  • Prudent Technologies and Consulting, Inc.

    Data Engineer

    Found in: Talent IN 2A C2 - 7 hours ago

    Prudent Technologies and Consulting, Inc. Hyderabad, India

    We are searching for an experienced Python Data Engineer with a strong focus on ETL processes and API development. This role is pivotal in designing and implementing efficient, scalable ETL pipelines and developing APIs to facilitate data access and manipulation. The ideal candid ...

  • ACL Digital

    Data Engineer

    Found in: Talent IN 2A C2 - 6 days ago

    ACL Digital Hyderabad, India

    Dear Folks, · We are hiring for the position of as AWS Data Engineer at Hyderabad location. · Only Interested and relevant candidates can share their resume at ) · Notice Period Required : Immediate to 15 Days Max · Designation : AWS Data Engineer · Experience Required : 3 to 12 ...

  • UWorld

    Data Engineer

    Found in: Appcast Linkedin IN C2 - 1 day ago

    UWorld Hyderabad, India

    UWorld is a worldwide leader in online test prep for college entrance, undergraduate, graduate, and professional licensing exams throughout the United States. Since 2003, over 2 million students have trusted us to help them prepare for high-stakes examinations. · We are seeking a ...

  • WaferWire Cloud Technologies

    Data Engineer

    Found in: Appcast Linkedin IN C2 - 7 hours ago

    WaferWire Cloud Technologies Hyderabad, India

    Hi, · This is Sundeep from Waferwire Technologies and we are hiring Data Engineer with Azure Synapses and Pyspark and ADF experience · Role: Data Engineer · Experience: 3-8 Years · Location: Hyderabad · Job Description: · We are seeking a seasoned Data Engineer to join our team i ...

  • Accellor

    Data Engineer

    Found in: Talent IN 2A C2 - 1 day ago

    Accellor Hyderabad, India

    Headquartered in the Silicon Valley with offices in London, Hyderabad and Singapore, Accellor is a Microsoft Gold Partner and a premier Salesforce Partner that uses best-of-breed cloud technology to deliver superior customer engagement and business effectiveness for clients. We b ...

  • Tech Mahindra

    Data Engineer

    Found in: Appcast Linkedin IN C2 - 7 hours ago

    Tech Mahindra Hyderabad, India

    Title: Data Operations Associate · Location: Hyderabad · Experience: 6+years · NP: Immediate to 15days · Company Profile: · Tech Mahindra represents the connected world, offering innovative and customer-centric information technology experiences. We #Rise together to create susta ...

  • Live Connections

    Data Engineer

    Found in: Appcast Linkedin IN C2 - 6 days ago

    Live Connections Hyderabad, India

    Hi All, · Hiring on Data Engineer - Full Time/Permanent – Hyderabad Location · Interested Applicants please share resumes to · Client: Confidential · Mandatory Skills: Data Engineer, Programming languages like Python, NoSQL, Spark, HDFS MapReduce, Cloud, AWS Glue, Azure Data Fa ...

  • Anblicks

    Data Engineer

    Found in: Appcast Linkedin IN C2 - 5 days ago

    Anblicks Hyderabad, India

    Position: Data Engineer - Snowflake · Experience: 5+ years · Location: Hyderabad · Anblicks is Data and AI Company – We bring value to your data. · Skills & Experience: · Experience in working with AWS S3/ Azure ADLS Storage Accounts and Snowflake. · Strong experience in data eng ...

  • SID Global Solutions

    Data Engineer

    Found in: Appcast Linkedin IN C2 - 7 hours ago

    SID Global Solutions Hyderabad, India

    Dear candidate, · Greetings · We are looking for Data Engineer role. Please find below the JD. If interested, please send me your updated resume to my email address: · Position: Data Engineer · Exp level: 5+ years · Skillset: SQL, AWS Stack, Python, Redshift and MYSQL · Roles & ...

  • Dreamz Consultants

    Data Engineer

    Found in: Talent IN 2A C2 - 4 days ago

    Dreamz Consultants Hyderabad, India

    Our client is a top rated global financial services powerhouse and is also a product company. We require people who can join within 45 days. · JD · Bachelor in computer science/engineering/statistics/economics (master degree preferred). · 7-8 years of experience in data analysis ...

  • Inspire Brands Hyderabad Support Center

    Data Engineer

    Found in: Appcast Linkedin IN C2 - 3 days ago

    Inspire Brands Hyderabad Support Center Hyderabad, India

    We are looking for a Master Data Management (MDM) Engineer for the Enterprise Data Organization to build and maintain MDM data products across multiple data domains that will support key data consolidation and governance initiatives. The candidate will be required to possess stro ...

  • C1 Global Innovation & Capability Center

    Data Engineer

    Found in: Talent IN 2A C2 - 7 hours ago

    C1 Global Innovation & Capability Center Hyderabad, India

    The Data Engineer's role is to work with the stake holders and project teams to ensure data needs/requirements are captured and fulfilled. This role would require a service-oriented mentality, high sense of ownership of the problems and requests assigned, focus on managing and re ...

  • Info Services

    Data Engineer

    Found in: Appcast Linkedin IN C2 - 4 days ago

    Info Services Hyderabad, India

    Data Engineer · Financial District, Hyderabad - Hybrid · Full-time/Permanent job · Work timing: From 12 pm to 9 pm IST · Must have strong Databricks, Spark, Scala, AWS, Airflow, Snowflake etc. · Minimum 4 years of related work experience is required. · For any query, please email ...

  • Zelis

    Data Engineer

    Found in: Talent IN 2A C2 - 7 hours ago

    Zelis Hyderabad, India Full time

    Responsibilities: · Design, develop, and maintain Azure Data Factory Pipelines with at least 5 years of experience, ensuring scalability, reliability, and efficiency. · Implement and optimize data pipelines in at least one Columnar MPP Cloud data warehouse (e.g., Snowflake, Azure ...

  • Growel Softech Pvt. Ltd.

    Data Engineer

    Found in: Talent IN 2A C2 - 10 hours ago

    Growel Softech Pvt. Ltd. Bangalore/Hyderabad, India permanent

    Job Description : · Role : Data Engineer · Required Technical Skills : · - More than 5 years' experience implementing data solutions · - Strong proficiency in Python with an emphasis in building data pipelines · - Experience in big data technologies like HDFS, YARN, Map-Reduce, K ...

  • PRUDENT GLOBALTECH SOLUTIONS PRIVATE LIMITED

    Data engineer

    Found in: Talent IN C2 - 5 days ago

    PRUDENT GLOBALTECH SOLUTIONS PRIVATE LIMITED Hyderabad, India Full time, CDD, Contract

    Company Overview · For over 25+ years, PRUDENT GLOBALTECH SOLUTIONS PRIVATE LIMITED has been a trusted provider of premium IT services, specializing in data sciences, cybersecurity, app development, and enterprise CRM. We work with best-of-breed technology firms and pride ourselv ...

  • PwC

    Data Engineer

    Found in: Talent IN C2 - 5 days ago

    PwC Hyderabad, India Full time

    Description · : GenAI Data Engineer - Associate PwC US - Acceleration Center is seeking an enthusiastic and emerging GenAI Data Engineer to join our team at the Associate level. This role is ideal for individuals passionate about data engineering and eager to develop their ski ...

  • Park Place Technologies

    Data Engineer

    Found in: Talent IN C2 - 5 days ago

    Park Place Technologies Hyderabad, India Full time

    Overview · The Data Engineering Team at Park Place Technologies is hiring talented engineers to help us build and ship innovative products and solutions with the latest technology stack to revolutionize our industry. · We seek self-motivated and results-driven individuals with a ...