Founding Data Engineer - New Delhi, India - CredHive

Description

Position Overview

We are a seed-funded startup focused on using state-of-the-art AI technologies to revolutionize the credit industry. Our team consists of experts in machine learning and software engineers who have worked at top-tier US tech companies like Apple, Amazon, etc , and we are passionate about using AI to improve access to credit information for businesses. We have the product on the market, the first clients, and sufficient runway. We are seeking a highly skilled and innovative Founding Data Engineer with a focus on web scraping and AWS data management. As a foundational member of our engineering team, you will play a crucial role in establishing and shaping our data infrastructure, focusing on extracting and managing large volumes of data essential for our services.

Key Responsibilities

• Web Scraping and Data Collection: Design, build, and maintain robust web scraping systems to collect data from various online sources.
• AWS Data Management: Set up and manage our data infrastructure on AWS, including services like S3, RDS, EC2, and AWS Lambda.
• Database Design and Management: Design and maintain scalable and efficient databases to store and process large datasets.
• Data Pipeline Architecture: Develop and manage end-to-end data pipelines, from data extraction and storage to processing and analysis.
• Quality Assurance: Implement systems to ensure the accuracy and integrity of data, including validation and cleansing processes.
• Optimization and Scaling: Continuously optimize data processes and infrastructure for performance and scalability.
• Collaborative Leadership: Work closely with other team members, providing guidance and support in data-related matters. Play a key role in strategic planning and decision-making processes.
• Innovative Problem-Solving: Stay ahead of the curve with emerging technologies and methodologies in data engineering and web scraping.

Qualifications

• Bachelor's or Master's degree in Computer Science, Engineering, or a related field.
• Strong experience in Python and web scraping tools and frameworks (such as BeautifulSoup, Scrapy, Selenium).
• Proficiency in managing and scaling data on AWS.
• Solid understanding of database technologies (SQL and NoSQL) and data warehousing.
• Familiarity with ETL processes and big data technologies.
• Proven ability to design and implement efficient data pipelines.
• Being scrappy
• Experience in a startup environment is a plus.

What We Offer

• Significant impact on the company's data strategy and infrastructure.
• A collaborative, flexible, and dynamic work environment.
• Competitive salary and benefits package.