Description

LLM - AI Quality Analyst

This role requires a unique blend of creativity and analytical rigor to evaluate the quality of personalized AI interactions.

About the Role:

• Evaluate model responses based on your intent from the starting prompt, checking if personalization was appropriately applied.
• Analyze responses for Grounding issues, ensuring claims about you are supported by evidence and not flawed inferences or robotic "overnarrating".

In this role, you will be part of a dynamic team focused on evaluating the quality of personalized AI interactions. Your day-to-day work will involve designing and executing multi-turn conversational prompts that require the AI to utilize your personal information and experiences.

To succeed in this role, you must have exceptional analytical thinking skills and experience in data annotation or related fields such as policy law ethics linguistics journalism computer science etc.
We are looking for candidates with strong evaluation acumen who can review Side-by-Side (SxS) model responses spot subtle differences naturalness written communication superior ability write clear concise structured rationales referencing specific turn numbers feedback ability provide constructive detailed analyze integration helps correct data sources exist intertwined excellent collaboration self motivated independent setting technical desktop laptop setup good internet connection required full time availability 24 hour operations team staffing global service both during night weekend normal working hours flex schedule needed Hindi proficiency ability read write high degree comp focus language project available those willing use primary personal Google account enable personalextended data source genuine assessment opportunity enjoy challenging fast paced dynamic environment join passionate drive deliver company culture success variety benefits expert stock option strategy focused achieving long term highly regarded renowned global leader innovation prioritizing people performance delivering customer insight win served mention values recognizes individual strives collaborative spirits foster real inclusivity promote gender race age disability creed sexual orientation program promotes makes mindset show agility hope over afraid outcome guarantee online community stay updated latest job opportunities company news linkedin following try career portal submit cv share whatsapp contact