Jobs
>
Hyderabad

    Pyspark Engineer5 YearsHyderabad - Notus Solution

    Notus Solution
    Notus Solution Hyderabad, India

    12 hours ago

    Default job background
    Description

    Overview:

    The PySparkEngineer plays a crucial role in our organization by leveragingtheir expertise in PySpark and Big Data technologies to designdevelop and maintain scalable data pipelines and analyticssolutions. This role is essential in enabling datadrivendecisionmaking and ensuring the optimal performance of our datasystems.

    KeyResponsibilities:

    • Designing andimplementing PySparkbased data processing and analyticssolutions.
    • Optimizing and tuning PySpark jobsfor performance and scalability.
    • Collaboratingwith data scientists and analysts to understand their requirementsand implement efficient dataworkflows.
    • Developing and maintaining ETLprocesses using PySpark to integrate data from multiplesources.
    • Creating and managing data pipelinesfor realtime and batchprocessing.
    • Troubleshooting and resolvingissues related to data processing and pipelineexecution.
    • Implementing best practices for dataengineering and ensuring data quality andreliability.
    • Collaborating with crossfunctionalteams to support datarelated initiatives andprojects.
    • Participating in code reviews andproviding technical guidance to junior teammembers.
    • Staying updated with the latestadvancements in PySpark and Big Datatechnologies.

    RequiredQualifications:

    • Bachelors or Mastersdegree in Computer Science Engineering or a relatedfield.
    • Minimum of 5 years of handson experiencein PySpark development and Big Datatechnologies.
    • Proficiency in Python programmingfor data analysis and manipulation.
    • Strongunderstanding of SQL and experience working with relationaldatabases.
    • Experience in building andoptimizing data pipelines for largescale dataprocessing.
    • Solid understanding of distributedcomputing principles and clustermanagement.
    • Knowledge of data warehousingconcepts and best practices.
    • Experience withcloud platforms such as AWS Azure or GCP is aplus.
    • Excellent problemsolving and analyticalskills with a detailoriented mindset.
    • Strongcommunication and collaboration abilities to work effectively in ateamenvironment.
    MandatorySkillData ModellerawsPysparkArchitect

    bigdata,python,sql,data engineering

  • Inspire

    Senior Analyst T500-10580

    4 days ago

    Inspire Hyderabad, India

    Job Summary:Reporting to the NOC Manager, HSC (Hyderabad Support Center), NOC (Network Operations Center), the Sr. Analyst has a responsibility for all technology aspects of Inspire Brands and will act as the primary technology expert for the NOC team. This role drives the mainte ...