- Developing notebooks and applying transformation using spark SQL in Azure data bricks.
- Testing the notebook and validating the scripts using SQL.
- Implemented activities Copy activity, Execute Pipeline, Get Meta data, If Condition, Lookup, Set Variable, Filter, For Each pipeline Activities for On-cloud ETL processing.
- Load the files to stage tables and then to data warehouse tables in dedicated SQL pool in synapse. Writing views, stored procedures for transformations.
- Converting views to external tables for direct query for faster retrieval.
- Preparing views data, data models for PBI reports Designed dashboards visuals with slicers, filters, drill through options.
- Assure that data is cleansed, mapped, transformed, and otherwise optimised for storage and use according to business and technical requirements Develop and maintain innovative Azure solutions Solution design using Microsoft Azure services and other tools The ability to automate tasks and deploy production standard code (with unit testing, continuous integration, versioning etc.)
- Load transformed data into storage and reporting structures in destinations including data warehouse, high speed indexes, real-time reporting systems and analytics applications Build data pipelines to collectively bring together data Other responsibilities include extracting data, troubleshooting and maintaining the data warehouse
-
Data Enginer
4 days ago
NatWest Group Bengaluru, Karnataka, IndiaOur people work differently depending on their jobs and needs. From hybrid working to flexible hours, we have plenty of options that help our people to thrive. · This role is based in India and as such all normal working days must be carried out in India. · Join us as a Data Engi ...
-
Data Engine
5 days ago
Decathlon Bengaluru, IndiaPermanent contract · - Full-time · - Mid-Senior level · - BTEC Higher National Diploma, Diploma of Higher Education · - Data engineer · **Mission**: · **Our Purpose**: · **_"Move people through the wonders of Sport"_** · **About Decathlon Sports India**: · Founded in 1976 in Fran ...
-
Data Engine
5 days ago
Decathlon India Bengaluru, India**Our Purpose**: *"Move people through the wonders of Sport"*About Decathlon Sports India: · Founded in 1976 in France, we have set about making the pleasure and benefit of sports available for everyone across the globe. We do that by providing world-class products both for enthu ...
-
Data Engineer t9
1 day ago
Mercedes-Benz Research and Development India Private Limited Bengaluru, India**Aufgaben**: · - Education: Masters/Bachelors in Computer Science, Electrical / Electronics or Equivalent engineering · Good analytics skills · - Skill Requirements: · - Excellent Programming skills in Python. · - Experience with cloud technologies like Azure, AWS, GCP, public o ...
-
Data Engineer-1
6 days ago
Commonwealth Bank Bengaluru, India**Your business**: · The Commonwealth Bank is the leading financial institution in Australia and provides integrated financial services. · The role sits within the Bankwest Technology division. Bankwest is a subsidiary of The Commonwealth Bank, and the Bankwest Technology divisio ...
-
Data Engineer 169368 Full-time
1 week ago
HyreFox Consultants Bengaluru, India Full timeBuild and maintain end-to-end data engineering pipelines on batch data. · - Design and implement batch data ingestion pipelines. · - Design and implement batch data transformation pipelines. · - Design and implement batch data storage solutions. · - Implement batch data governanc ...
-
Azure Data Engineer-3-10years-hyderabad
1 week ago
Black & White Business Solutions Bengaluru, India**Job Information**: · Industry · **IT Services** · *** · Province · **Karnataka** · *** · City · **Bangalore North** · *** · Postal Code · **560001** · *** · Country · **India** · Job Title:Azure data Bricks · Years of EXP:3 to 10Years · Location:Hyderabad · NP:30Days/Immediate ...
-
Data Engineer
1 week ago
Joveo Bengaluru, IndiaWe are looking for talented Data Engineers who are passionate about data engineering and have strong experience in building scalable data pipelines. As a Data Engineer at Joveo, you will play a leading role in designing and building datasets that power analytics for teams across ...
-
Data Engineer
1 week ago
Ascendion Bengaluru, IndiaJob Title: Data Engineer/ Lead · Duration: Full Time · Location: Bengaluru, Hyderabad, Chennai, Pune (Hyrbrid) · NOTE: Looking for Immediate Joiners ( 2-3 weeks notice) · Proficient in Azure Cloud services (or GCP is Ok) for data engineering tasks. · Familiarity with Kubeflow, wi ...
-
Data Engineer
1 day ago
Ascendion Bengaluru, IndiaJob Title: Data Engineer · Job Location: Bangalore (Remote) · Experience Level: 5-8 Years · Job Summary: We are seeking an experienced Data Engineer with a strong background in SQL Server and Azure Data Factory to join our team. The ideal candidate will be responsible for designi ...
-
Engineer, Data
6 days ago
Giant Eagle GCC , Karnataka, IndiaJob Summary · As a Data Engineer on the Customer Loyalty & Experience team, you will be working on a team to bring customer-centric personalization to life. In this role, you will be empowered to develop data solutions in support of marketing analytics and data science by engagin ...
-
Data Engineer
4 days ago
Recro Bengaluru, IndiaExperience:3+ years · Work location: Bangalore · primary_skills(Must_have) · Programming Skills: Proficiency in Python, Pyspark, and SQL is essential. · Big Data Experience: Hands-on experience with Spark and relational databases such as Postgres is required. · Orchestration To ...
-
Data Engineer
3 days ago
LeadSquared Bengaluru, IndiaData Engineer: · About the Role: We are looking for a data engineer to work on building applications to migrate data from various data sources and building analytics engine to cater to product's KPI requirements along with some knowledge on data science concepts to implement in e ...
-
Data Engineer
1 week ago
Indium Software Bengaluru, IndiaTitle: Data Engineer · Experience: 2-6/7 Years · Work Location: Bengaluru/Hyderabad · Work Mode: Work from Office · Skills: Adv SQL, python, Spark or pyspark, Hive, Airflow, ETL, Good Communication skills, Business acumen , Team handling experience. · Roles and Responsibilities: ...
-
Data Engineer
1 day ago
Axtria - Ingenious Insights Bengaluru, IndiaJob Title: Data Engineer / ETL Developer · Job Description: · We are seeking an experienced ETL Developer / Data Engineer to join our team. The ideal candidate will have a strong background in Azure Data Factory (ADF), SQL, Pyspark/Databricks · Responsibilities: · Design, develo ...
-
Data Engineer
1 week ago
Tech Mahindra Bengaluru, IndiaRole: GCP Data Engineer · Location: Hyderabad/ Bengaluru Work from office (Mon-Fri) · Duration: Full Time · Required Technical Skills: · ● More than 7 years' experience implementing data solutions · ● Strong proficiency in Python with an emphasis in building data pipelines · ● Ex ...
-
Data Engineers
1 week ago
Multplyr Bengaluru, IndiaPayroll Company: Multplyr · Role: Data Engineers · Experience: years, across levels · Location: Hybrid / WFO at our client offices in Bengaluru, Mumbai, Pune, Delhi/NCR, Hyderabad and Chennai · Type: Contract / Contract To Hire · Typical Duration: months · About Us · Multplyr is ...
-
Sr Data Engineer/Data Engineer
1 week ago
Trigent Software - Professional Services Bengaluru, IndiaSenior Data Engineer (5+ Years) / Data Engineer (3+): · Experience working with distributed technology tools for developing Batch and Streaming pipelines using · SQL, Spark, PySpark, · Airflow [3+ years], · Spark with Scala (optional). · Able to write code which is optimized ...
-
Data Engineer
3 days ago
JRD Systems Bengaluru, IndiaRole : Data Engineer - AWS and Oracle Expert · Job Summary: · We are seeking a highly skilled and experienced Data Engineer with expertise in data development in AWS and Oracle. The ideal candidate will have a minimum of 5 years of experience in data engineering, with a strong fo ...
-
Data Engineer
1 week ago
Course5i Bengaluru, IndiaThe Data Engineer will be responsible for designing, building, and maintaining data pipelines and infrastructure that enable efficient and accurate data processing. · The Data Engineer will work closely with data scientists, analysts, and project teams to ensure data is accessibl ...
Azure Data Engineer - Bengaluru, India - Tech Mahindra
Description
Designed Azure ADF pipelines to move data from Open hubs to Azure Data Lake Gen2 and then to Azure Data Warehouse.
Role: Data Engineer (4-8 year relevant exp)
Primary skills:Advance SQL/DWH, Strong is SQL hands on experience in writing complex queries/stored procedure, Azure Data Factory, Azure Data Bricks, ADLS)