Jobs
>
Bengaluru

    Azure Data Engineer - Bengaluru, India - Fractal

    Fractal
    Fractal Bengaluru, India

    6 days ago

    Default job background
    Upper Management / Consulting
    Description

    JOB DESCRIPTION:

    If you are an extraordinary developer and who loves to push the boundaries to solve complex business problems using creative solutions, then we wish to talk with you. As a Data Engineer - Azure, you will work in the Technology team that helps deliver our Data Engineering offerings at large scale to our Fortune clients worldwide. The role is responsible for innovating, building, and maintaining technology services.

    RESPONSIBILITIES:


    • Be an integral part of large-scale client business development and delivery engagements.


    • Develop the software and systems needed for end-to-end execution on large projects.


    • Work across all phases of SDLC and use Software Engineering principles to build scaled solutions.


    • Build the knowledge base required to deliver increasingly complex technology projects.


    • Developing Data Pipelines (Batch/Streaming)


    • Developing Complex data transformations


    • ETL Orchestration


    • Data Migration


    • Develop and Maintain Datawarehouse / Data Lakes

    EXPERIENCE:


    • Above 7 years of experience in setting up data pipelines to injest streaming and batch data from many sources


    • Experience on optimized data collection procedures


    • Good experience in performing extract, transform and load (ETL) processes to make the data more usable with cloud infrastructures


    • Expertise in data storage structures, data mining, and data cleansing


    • Hands-on experience using Big Data components (Hadoop, Spark) & programming languages.


    • Experience in using, or migrating to cloud databases


    • Proficient in relational database design and development


    • Proficient in Data warehousing concepts and up to date Data Integration patterns/technologies


    • Experience in Azure data lake, Azure SQL Database, Data Bricks, Azure SQL Data warehouse and Azure Data factory.


    • Experience in Developing Spark applications using Spark - SQL in Databricks for data extraction,     transformation, and aggregation from multiple file formats for analyzing & transforming the data to uncover insights.


    • Systematic problem-solving approach with strong communication skills and a sense of ownership and drive

    TECHNICAL SKILLS:


    • Strong Experience in Azure Cloud


    • Experience with distributed data/computing tools: Map/Reduce, Hadoop, Hive, Spark


    • Strong knowledge in Azure Analytics Products like Data Factory, HD Insights, Azure Data Lake Storage, Azure SQL Datawarehouse & Data Bricks


    • Strong in Python Programming Skills


    • Strong experience in networking and security measures


    • Good knowledge in using NoSQL databases like MongoDB, PostgreSQL and Cassandra running huge volumes of data


    • Proficient in with SQl Databases like MSSql, Postgres.


    • Expertise in Data Warehousing – Hadoop, MapReduce, HIVE, PIG, Apache Spark, Kafka


    • Good understanding of Spark Architecture including Spark Core, Spark SQL, Data Frames, Spark Streaming, Driver Node, Worker Node, Stages, Executors and Tasks.


    • Good understanding of Big Data Hadoop and Yarn architecture along with various Hadoop Demons such as Job Tracker, Task Tracker, Name Node, Data Node, Resource/Cluster Manager, and Kafka (distributed stream-processing).

    JOB RESPONSIBILITES:


    • Building and maintaining the data architecture for data science projects.


    • Extract Transform and Load data from sources Systems to Azure Data Storage services using a combination of Azure Data Factory, T-SQL, Spark SQL and U-SQL Azure Data Lake Analytics.


    • Data Ingestion to one or more Azure Services - (Azure Data Lake, Azure Storage, Azure SQL, Azure DW) and processing the data in In Azure Data bricks.


    • Define table schema and quickly adapt with the pipeline.


    • Improve data foundational procedures, integrating new data management technologies and softwares into the existing system, building data collection pipelines


    • Identify valuable data sources and automate collection processes.


    • Implement concepts of programming such as data structures, error handing, data manipulation and I/O processing. Implements database concepts and practices including definition and query language.


    • Implement adequate networking and security measures for the data pipeline.


    • Implement monitoring solution for the data pipeline.


    • Develop Spark applications using Pyspark and Spark-SQL for data extraction, transformation and aggregation from multiple file formats for analyzing & transforming the data to uncover insights into the customer usage patterns.


    • Support the design, and implement data engineering solutions


    • Maintain excellent documentation for understanding and accessing data storage


    • Work independently as well as in teams to deliver transformative solutions to clients


    • Be proactive and constantly pay attention to the scalability, performance and availability of our systems


    • Establishes privacy/security hierarchy and regulates access


    • Undertake preprocessing of structured and unstructured data


    • Support in collecting, organizing, and interpreting data along with fellow colleagues.


    • Collaborate with engineering and product development teams

    EDUCATION:


    • B.E/B.Tech/M.Tech in Computer Science or related technical degree OR Equivalent

  • Decathlon India

    Data Engine

    4 days ago

    Decathlon India Bengaluru, India

    **Our Purpose**: *"Move people through the wonders of Sport"*About Decathlon Sports India: · Founded in 1976 in France, we have set about making the pleasure and benefit of sports available for everyone across the globe. We do that by providing world-class products both for enthu ...

  • NatWest Group

    Data Enginer

    3 days ago

    NatWest Group Bengaluru, Karnataka, India

    Our people work differently depending on their jobs and needs. From hybrid working to flexible hours, we have plenty of options that help our people to thrive. · This role is based in India and as such all normal working days must be carried out in India. · Join us as a Data Engi ...

  • Decathlon

    Data Engine

    4 days ago

    Decathlon Bengaluru, India

    Permanent contract · - Full-time · - Mid-Senior level · - BTEC Higher National Diploma, Diploma of Higher Education · - Data engineer · **Mission**: · **Our Purpose**: · **_"Move people through the wonders of Sport"_** · **About Decathlon Sports India**: · Founded in 1976 in Fran ...

  • Commonwealth Bank

    Data Engineer-1

    4 days ago

    Commonwealth Bank Bengaluru, India

    **Your business**: · The Commonwealth Bank is the leading financial institution in Australia and provides integrated financial services. · The role sits within the Bankwest Technology division. Bankwest is a subsidiary of The Commonwealth Bank, and the Bankwest Technology divisio ...

  • HyreFox Consultants

    Data Engineer 169368 Full-time

    1 week ago

    HyreFox Consultants Bengaluru, India Full time

    Build and maintain end-to-end data engineering pipelines on batch data. · - Design and implement batch data ingestion pipelines. · - Design and implement batch data transformation pipelines. · - Design and implement batch data storage solutions. · - Implement batch data governanc ...

  • Black & White Business Solutions

    Azure Data Engineer-3-10years-hyderabad

    6 days ago

    Black & White Business Solutions Bengaluru, India

    **Job Information**: · Industry · **IT Services** · *** · Province · **Karnataka** · *** · City · **Bangalore North** · *** · Postal Code · **560001** · *** · Country · **India** · Job Title:Azure data Bricks · Years of EXP:3 to 10Years · Location:Hyderabad · NP:30Days/Immediate ...

  • UNO Digital Bank

    Data Engineer

    6 days ago

    UNO Digital Bank Bengaluru, India

    Overview: · We are looking for an experienced data engineer to join our team. You will use various methods to transform raw data into useful data systems. Overall, you'll strive for efficiency by aligning data systems with business goals. · To succeed in this data engineering po ...

  • Informatica

    Data Engineer

    1 week ago

    Informatica Bengaluru, India

    Build Your Career at Informatica · We're looking for a diverse group of collaborators who believe data has the power to improve society. Adventurous minds who value solving some of the world's most challenging problems. Here, employees are encouraged to push their boldest ideas f ...

  • SMARTBORN SOFTWARE CONSULTING PVT.

    Data Engineer

    3 days ago

    SMARTBORN SOFTWARE CONSULTING PVT. Bengaluru, India

    Role Description · This is a full-time hybrid role as a Data Engineer at SMARTBORN SOFTWARE CONSULTING PVT.LTD. The Data Engineer will be responsible for day-to-day tasks related to data engineering, including data modeling, Extract Transform Load (ETL) development, data warehous ...

  • Quantum Labs

    Data Engineer

    5 days ago

    Quantum Labs Bengaluru, India

    Company Description · Quantum Labs is a product and technology consulting studio specializing in web, mobile, and AI applications. With a team of experienced professionals, we are committed to delivering high-quality services that meet the unique needs of our clients. At Quantum ...

  • TEKsystems

    Data Engineer

    3 days ago

    TEKsystems Bengaluru, India

    Job Description: · We are seeking a skilled and motivated individual to join our team as a Snowflake Data Engineer with expertise in Python development. As a key member of our data team, you will play a crucial role in designing, implementing, and optimizing data pipelines withi ...

  • Giant Eagle GCC

    Engineer, Data

    5 days ago

    Giant Eagle GCC , Karnataka, India

    Job Summary · As a Data Engineer on the Customer Loyalty & Experience team, you will be working on a team to bring customer-centric personalization to life. In this role, you will be empowered to develop data solutions in support of marketing analytics and data science by engagin ...

  • Razorpay

    Data Engineer

    4 days ago

    Razorpay Bengaluru, India

    The Role: · We are looking for a savvy Data Engineer to join our growing team, responsible for expanding and optimizing our data and data pipeline architecture, as well as optimizing data flow. The ideal candidate is an experienced data pipeline builder and data wrangler who enjo ...

  • Recro

    Data Engineer

    3 days ago

    Recro Bengaluru, India

    Experience:3+ years · Work location: Bangalore · primary_skills(Must_have) · Programming Skills: Proficiency in Python, Pyspark, and SQL is essential. · Big Data Experience: Hands-on experience with Spark and relational databases such as Postgres is required. · Orchestration To ...

  • Tavant

    Data Engineer

    1 week ago

    Tavant Bengaluru, India

    With 24+ years of experience building innovative digital products and solutions, Tavant provides impactful results to its customers. It has been the frontrunner in driving digital innovation and tech-enabled transformation across a wide range of industries such as Fintech, Manufa ...

  • Datafoundry

    Data Engineer

    36 minutes ago

    Datafoundry Bengaluru, India

    Job Role - Data Engineer · Experience: 4-7 years · Location – Bangalore / Hyderabad / Visakhapatnam · Work Mode – On-site / Work from office (5 working days) · About Datafoundry AI: · We're Datafoundry AI, a dedicated team of data scientists, life sciences experts, and technol ...

  • Trigent Software - Professional Services

    Sr Data Engineer/Data Engineer

    1 week ago

    Trigent Software - Professional Services Bengaluru, India

    Senior Data Engineer (5+ Years) / Data Engineer (3+): · Experience working with distributed technology tools for developing Batch and Streaming pipelines using · SQL, Spark, PySpark, · Airflow [3+ years], · Spark with Scala (optional). · Able to write code which is optimized ...

  • JRD Systems

    Data Engineer

    1 day ago

    JRD Systems Bengaluru, India

    Role : Data Engineer - AWS and Oracle Expert · Job Summary: · We are seeking a highly skilled and experienced Data Engineer with expertise in data development in AWS and Oracle. The ideal candidate will have a minimum of 5 years of experience in data engineering, with a strong fo ...

  • C5i

    Data Engineer

    36 minutes ago

    C5i Bengaluru, India

    OVERVIEW · Course5 Intelligence · We enable organizations to make the most effective strategic and tactical moves relating to their customers, markets, and competition at the rapid pace that the digital business world demands. Founded in 2000, our business areas include Market ...

  • Ecolab Digital Center

    Data Engineer

    1 week ago

    Ecolab Digital Center Bengaluru, India

    3+ Years professional experience in a data Engineer/analyst role or similar. · o At least 1 year of Development experience in ADF V2 . · o At least 1 years of Experience with Spark (PySpark and other python data libraries), Spark Scala is also fine. · o At least 1 years of experi ...