Assess AI-generated multilingual reference answers and evaluation rubrics to support the development of advanced AI language models. Use precise guidelines to identify quality issues and ensure accurate scoring standards.
About this project
This project involves reviewing both open-ended and closed-ended AI responses and their associated scoring rubrics to maintain consistent and high-quality evaluation across multiple languages.
Responsibilities
- Analyze AI-produced reference responses for accuracy and quality.
- Review and refine evaluation rubrics to improve scoring consistency.
- Detect and correct critical errors impacting evaluation quality.
- Adhere strictly to detailed annotation guidelines and quality standards.
Requirements
- Native-level proficiency in English, Indonesian, Malay, Portuguese, Spanish, Thai, or Vietnamese.
- Excellent reading comprehension and written communication skills.
- Basic knowledge of medical, financial, and legal terminology.
- Strong attention to detail and ability to follow complex instructions.
- Access to a computer with a stable internet connection.
Project details
- Location: Remote work available in Brazil, Indonesia, Malaysia, Mexico, Thailand, United Kingdom, and one additional eligible location.
- Schedule: Flexible hours; availability of tasks may vary.
Compensation
Payment is provided at a fixed hourly rate, which varies depending on the selected language and location.