Multilingual RLHF Quality Reviewer

What you will do

• Review annotator preference judgements, ratings and written responses for correctness, consistency with guidelines and freedom from bias.
• Adjudicate disagreements between annotators with a documented rationale.
• Audit sampled batches and report quality metrics against programme thresholds.
• Identify systematic errors, whether they originate in the guidelines, in training or in an individual reviewer, and specify which.
• Give annotators feedback that measurably improves their next batch.
• Contribute to guideline refinement where the current version is producing inconsistent outcomes.

What you bring

• English at C1 plus one of Arabic, Persian, Urdu, Turkish or French at C1 or above.
• At least two years in annotation, linguistic quality, editorial work, translation review, teaching or a comparable judgement-based discipline.
• Ability to reason carefully about ambiguous cases and to write down why you decided as you did.
• Sound judgement on safety, bias and factual accuracy.
• Total reliability on deadlines, since a delayed audit blocks a training run.

Nice to have

• Prior RLHF or model evaluation experience.
• A postgraduate qualification in linguistics, philosophy, law or a scientific field.
• A third language.
• Subject matter depth in medicine, law, finance or engineering.

What we offer

• Fully remote and flexible within weekly minimums.
• Rates that increase with review tier and language scarcity.
• Paid guideline training for each programme.
• Rolling contracts with steady volume across programmes.
• Work that directly shapes model behaviour, which is rare and does not stay rare for long.

How the process works

Apply on this page. The first step takes under two minutes. Every application gets a response from our talent team.