LLM - AI Quality Analyst (Personalization) - Spanish

LLM - AI Quality Analyst (Personalization) - Spanish Remote (Spain)ContractView all jobsRole OverviewWe’re hiring an AI Quality Analyst to help us evaluate a personalization feature we’re building into Gemini. The idea behind it is pretty straightforward, the model should be able to use what it knows about you (past conversations, Gmail, Search, YouTube activity) to give you answers that actually feel relevant, not just technically correct.Your job is to put that to the test. You’ll come up with prompts based on your own experiences, run them through the model, and then honestly assess whether the responses felt personalized in a meaningful way or just kind of generic with a personal detail tacked on. It’s equal parts creative and analytical, and the quality of your judgment really does matter here.ResponsibilitiesDesign multi-turn conversational prompts (typically 1–5 turns) that require the AI to draw on real personal information and experiences.Evaluate whether the model applied personalization correctly based on what was actually being asked.Review responses for Grounding issues. Flag anything that looks like a flawed inference or hallucination rather than evidence-backed reasoning.Assess Integration quality. Does the personal data feel naturally woven in, or does it come across as robotic and forced?Stack-rank two model responses side-by-side (SxS) based on helpfulness, ease of use, and overall quality.Write clear, well-structured rationales that reference specific turns in the conversation.Extract and verify “Debug Info” to confirm chat summaries and data sources were properly used.Clear evaluation conversations after each session to maintain clean data.Required QualificationsStrong English reading and writing skills, the project is conducted entirely in English.Demonstrated ability to evaluate nuanced or ambiguous AI responses and explain your reasoning clearly.Comfortable working independently in a remote setup with minimal hand-holding.Reliable desktop or laptop with a stable internet connection.Full-time availability in your local time zone with at least 4 hours of daily overlap with PST.Preferred QualificationsExperience in data annotation, AI quality evaluation, content moderation, or something similar.BS/BA degree or equivalent experience in a relevant field - Policy, Law, Ethics, Linguistics, Journalism, Computer Science, or anything analytically rigorous.Familiarity with personalization concepts and a good instinct for spotting bad inferences or forced connections.Experience designing prompts or testing AI systems in any capacity.Sharp attention to detail when comparing side-by-side responses, especially around tone and naturalness.Ability to write feedback that’s specific and actionable, not just general impressions.Engagement DetailsThis is a contractor role starting immediately. We’re running a 24-hour global operation, so schedule consistency matters. There are two commitment options:• 30 hours/week - at least 4 hours per day, with a minimum 4-hour overlap with PST.• 40 hours/week - same daily and overlap requirements.Hiring ProcessThere are three steps to complete before being considered: • Screener • Three assessments • Language vettingShortlisted candidates will receive a Job Interest Form first. Once your profile is reviewed, you’ll have 24 hours to complete an assessment. From there, we’ll get in touch with finalists to go over pre-onboarding requirements.ApplyAutofill from resumeDrag & drop or upload to autofill your application.Upload fileFirst Name *​Last Name *​Email *​LinkedIn Profile URL *​Phone Number *​The salary expectation for this is $10/hr. Will you be comfortable with that? *YesNoAre you located in Spain? *YesNoWhat's your current residence? *​Are you legally authorized to work as an independent contractor in Spain? *YesNoWhat is your English proficiency level? *NativeFluentProfessional Working ProficiencyConversationalResume *Drag and drop file or browse computer(PDF, 5 MB...