Senior RLHF Evaluator

We need evaluators who can read AI-generated text critically and make consistent quality judgments under detailed rubrics. On a given day you might compare two explanations of quantum mechanics, two pieces of marketing copy, and two responses to a sensitive personal question. The common thread is careful reading, rubric application, and clear written justifications. Speed matters, but not at the expense of thoughtfulness.

Ideal candidates have a graduate-level education or equivalent professional experience in a field that requires careful argumentation and evidence evaluation. We have had especially strong results from people with backgrounds in academic research, technical writing, legal analysis, and scientific peer review. Familiarity with LLM failure modes -- hallucination, sycophancy, refusal errors -- is valuable.

Onboarding takes about one week and includes rubric training, practice tasks with feedback, and a calibration exam. After onboarding, you work asynchronously on your own schedule. A Slack workspace provides real-time access to project leads and fellow annotators for guideline questions.