AI Evaluation / Rubric Writer (Remote, Part‑time)

About the role

The AI Evaluation/Rubric Writer is a part‑time, remote position based in Addis Ababa. You will design, write, and refine evaluation rubrics and test scenarios that assess AI‑generated content, chat interactions, and recommendation quality, ensuring alignment with hospitality standards and guest safety.

Key responsibilities

• Research user needs and industry practices to draft objective evaluation frameworks.
• Create clear rubrics, test scenarios, and scoring criteria for AI outputs.
• Test AI responses against rubrics, analyze results, and produce concise quality reports.
• Collaborate with product, engineering, and customer‑support teams to improve AI performance and suggest prompt or policy enhancements.

Required profile

• Strong writing and editing abilities for clear rubric instructions and reports.
• Analytical and research skills to design objective criteria and incorporate hospitality industry standards.
• Basic understanding of AI/ML concepts and interest in AI ethics and safety.
• Bachelor’s degree in Linguistics, Computer Science, Hospitality Management or related field, or equivalent experience.

Required skills

• Proficiency with digital documentation tools and spreadsheets.
• Ability to learn new online platforms quickly.