Data Engineer - Bengaluru, India - PricewaterhouseCoopers Service Delivery Center (Bangalore) Private Limited

Description

:
GenAI Data Engineering - Senior Manager

PwC US - Acceleration Center is seeking a seasoned and visionary GenAI Data Engineer to join our team at the Senior Manager level.

In this leadership role, you will oversee the development and enhancement of data pipelines, champion the implementation of machine learning models, and drive the optimization of our data infrastructure for GenAI projects.

The ideal candidate will have extensive experience in data engineering with a deep focus on GenAI technologies, coupled with an advanced understanding of data processing, event-driven architectures, containerization, and cloud computing solutions.

Responsibilities:
Lead the strategic planning and development of advanced data pipelines and ETL processes tailored for GenAI projects.

Direct and mentor teams of data scientists and software engineers in the implementation and optimization of machine learning models and algorithms.

Spearhead the optimization and scaling of data infrastructure and storage solutions to support efficient data processing across diverse GenAI applications.

Design and implement robust event-driven architectures to facilitate real-time data processing and analytics.

Oversee the deployment and management of containerization technologies like Kubernetes and Docker, ensuring high scalability and efficiency.

Manage the creation and maintenance of expansive data lakes for effective handling of large-scale structured and unstructured data.

Guide the integration and practical application of LLM frameworks (Langchain, Semantic Kernel) to enhance natural language processing and analysis capabilities.

Lead cross-functional teams in designing and executing comprehensive solution architectures for GenAI initiatives.

Champion the use of cloud computing platforms such as Azure or AWS to streamline data processing, storage, and deployment efforts.

Conduct high-level monitoring and troubleshooting of data pipelines and systems to guarantee continuous and reliable data flow.

Stay abreast of the latest developments in GenAI technologies and lead the adoption of cutting-edge solutions to refine data engineering practices.

Facilitate the translation of complex business requirements into scalable technical solutions, enhancing data-driven decision-making processes.

Develop and enforce documentation standards for data engineering processes, methodologies, and best practices.

Maintain and advance professional knowledge through ongoing certification and training in solution architecture and industry best practices.

Requirements:
Bachelor's or Master's degree in Computer Science, Data Science, or a related field.

13+ years of relevant technical/technology experience, with significant expertise in GenAI projects.

Advanced programming skills in Python and fluency in data processing frameworks like Apache Spark.

Extensive experience with SQL, database management systems, and data architecture principles.

Proven track record with event-driven architectures and real-time data processing systems.

In-depth knowledge of containerization technologies such as Kubernetes and Docker.

Comprehensive understanding of data lakes and strategies for data management.

Expertise in LLM frameworks including Langchain and Semantic Kernel.

Proficiency in cloud computing platforms such as Azure or AWS.

Exceptional leadership, problem-solving, and analytical abilities.

Superior communication and collaboration skills, with experience managing high-performing teams.

Ability to operate effectively in a dynamic, fast-paced environment.

Nice to Have Skills:

Experience with additional technologies such as Databricks, Azure AI Search, Azure OpenAI, Azure Event Hub, Azure Data Lake Storage, AWS Open Search, AWS Bedrock, AWS Event Bridge, AWS S3, Azure Key Vault, DataDog, and Splunk.

Possession of relevant solution architecture certificates and continuous professional development in data engineering and GenAI.

Join PwC US - Acceleration Center as a Senior Manager in GenAI Data Engineering to lead transformative projects and drive innovation in a collaborative and forward-thinking environment.

We offer a competitive compensation package and significant opportunities for career advancement and professional growth.


EducationDegrees/Field of Study required:

Degrees/Field of Study preferred:
CertificationsRequired SkillsOptional SkillsDesired LanguagesTravel RequirementsNot Specified

Available for Work Visa Sponsorship?No

Government Clearance Required?No

Job Posting End Date