Generative AI Quality Analyst Overview

last updated June 6, 2026 19:11 UTC

Welo Data

HQ: On-site

more jobs in this category:

  • -> Transcriptionist @ Literably
  • -> Remote Freelance Writers for Entry-Level Blogging Work @ We Write Blog Posts
  • -> English Transcript Editor @ 3Play Media
  • -> Data Entry Operator @ Tower Health
  • -> Data Entry specialist @ RMJ4U

Welo Data partners with technology companies to deliver high-quality, ethically sourced datasets that are relevant, diverse, and scalable to help accelerate their AI models. As a Welocalize brand, WeloData draws on more than 25 years of experience supporting the world’s most innovative companies and brings together a curated global community of 500,000+ AI training and domain experts to offer services across:

ANNOTATION & LABELLING: Transcription, summarization, image and video classification, and labeling.
ENHANCING LLMs: Prompt engineering, SFT, RLHF, red teaming, adversarial model training, and model output ranking.
DATA COLLECTION & GENERATION: From institutional languages to audio collection from remote fields.
RELEVANCE & INTENT: Culturally nuanced and context-aware ranking, relevance, and evaluation for training models used in search, ads, and LLM outputs.

Want to join the Welo Data team? We bring practical, hands-on AI expertise to our projects, combining strong academic experience with deep working knowledge of state-of-the-art AI tools, frameworks, and best practices. Help us elevate our clients’ data at Welo Data.

Role Purpose
Owns client-specific quality programs, performance metrics, and escalations, while leading team development and coordinating across functions. Shares accountability for client outcomes and team performance with the Ops Manager.

Key Responsibilities
– Define client QA strategies (sampling design, audit methods, acceptance thresholds) and adjust them to scope and volume changes.
– Conduct root-cause analyses; drive CAPA plans with owners, timelines, and effectiveness checks.
– Plan training and certification for raters/annotators and coordinators; monitor completion and impact.
– Maintain dashboards (throughput, accuracy, productivity, and cost) and turn insights into actions.
– Manage client escalations by presenting options, trade-offs, and recovery paths.
– Standardize SOPs, templates, and checklists to eliminate bottlenecks.
– Pilot small automations (macros, templates, RPA/API handoffs) with Ops Tech, then scale what works.
– Coach P1s and C2–C3 on tools, workflows, and QA practices.
– Ensure compliance and security in data handling and platform access.

Requirements
Education
– Bachelor’s degree or equivalent experience in Business, Operations, Quality, or Data/Engineering.

Experience
– 2+ years in quality/operations with hands-on QA and workforce/training coordination.
– 1+ years leading people or pods (formally or informally).

Skills
– Strong multi-project planning and stakeholder management.
– Clear client communications and participation in governance cadence.
– Strong skills with spreadsheets, PM/task boards, and basic BI; ETL knowledge is a plus.
– Capacity planning with vendors; confident escalation and negotiation.
– Ability to work effectively in global, distributed teams.

Additional Qualifications
– Near-native English with strong writing and editorial skills.
– Hands-on experience with generative AI tools (text/voice/video).
– Background in QA testing, rubric design, or AI safety/ethics evaluation.
– Familiarity with data-annotation platforms and model-evaluation tools.
– Ability to interpret code/datasets/workflows at a conceptual level (no coding required).
– Works independently and manages workflows effectively in a remote setting.
– Multilingual ability beyond English.

Scope & Autonomy
– Leads quality, training, and performance tracks across multiple projects; operates independently within defined guardrails.
– Shares accountability for client results and team performance with the Ops Manager.

Job Reference: #LI-JC1

We may use artificial intelligence (AI) tools to support parts of the hiring process, such as reviewing applications, analyzing resumes, or assessing responses. These tools assist our recruitment team but do not replace human judgment. Final hiring decisions are ultimately made by humans. If you would like more information about how your data is processed, please contact us.

Apply info ->

To apply for this job, please visit the application page

Shopping Cart
There are no products in the cart!
Total
 0.00
0