Job Description
We are sharing a specialised part-time consulting opportunity for bilingual professionals fluent in English and Italian who are interested in contributing to the evaluation and improvement of advanced AI systems.
This role supports collaborations with leading AI teams focused on improving how conversational AI models respond to real-world user questions across a wide range of topics.
Experts will evaluate AI-generated responses and provide structured feedback to ensure accuracy, clarity, reasoning quality, and natural communication aligned with human expectations.
Key Responsibilities
Evaluate AI-generated responses to determine how effectively they answer user queries
Conduct fact-checking using trusted public sources and external tools
Annotate responses by identifying strengths, weaknesses, and factual inaccuracies
Assess reasoning quality, tone, clarity, and completeness of responses
Ensure model responses align with expected conversational behavior and system guidelines
Apply consistent evaluation standards using structured taxonomies and benchmarks
Ideal Profile
Strong candidates will have:
Bachelors degree or higher
Language Requirements:
Native or C2-level proficiency in Italian
Fluent English
Additional qualities include:
Experience working with large language models (LLMs)
Excellent writing and feedback articulation skills
Strong analytical and critical thinking ability
High attention to detail and ability to detect subtle reasoning issues
Comfort working across diverse topics and domains
Additional Background That Helps
Candidates often come from fields requiring structured analytical thinking, such as:
Research
Policy analysis
Data analytics
Linguistics
Engineering
Strong college-level mathematics and reasoning skills are also beneficial for the role.
Nice-to-Have Experience
Experience with RLHF, model evaluation, or AI data annotation
Experience producing high-quality written content
Experience comparing multiple outputs and making qualitative judgments
Familiarity with evaluation rubrics or structured scoring frameworks
Location Requirements
Candidates should be based in one of the following regions:
Europe
United States
What Success Looks Like
Identifying factual inaccuracies and reasoning errors in model outputs
Producing clear and consistent evaluation feedback
Delivering reproducible annotations that improve AI system performance
Helping ensure AI systems communicate clearly and reliably
Why This Opportunity
Contribute directly to improving how AI systems interact with real users
Work with cutting-edge AI labs and research teams
Flexible remote work with competitive compensation
Play a direct role in shaping next-generation AI systems
Contract Details
Independent contractor role
Fully remote with flexible scheduling
Compensation: $36.16/hour
Weekly payments via Stripe or Wise
Projects may extend depending on performance and project needs
About the Platform
This opportunity is available through a leading AI-driven work platform.
...Job Description Job Description We are seeking dedicated Contract Assessment Nurses for an excellent opportunity! Flexible Schedule Available Great Earning Potential Non-Bedside Are you interested in home health services? Do you crave a flexible work...
Health Fitness Professional (part-time) page is loaded## Health Fitness Professional (part-time)locations: Purchase, NYtime type: Part timeposted on: Posted Todayjob requisition id: 104864HealthFitness, a Trustmark company, is a proven leader in providing fitness solutions...
A global sportswear company seeks a Professional Affiliate Marketing specialist in Raleigh, North Carolina. In this role, you will manage affiliate program relationships, oversee marketing placements, and ensure smooth operations. A bachelor's degree and related experience...
...maintain a safe, healthy learning environment, including full supervision of children at all times, in accordance with Head Start and childcare licensing requirements. 28. Conduct daily health/safety monitoring of the indoor and (as requested) outdoor environment. 2...
...Services Northbridge , MA , USPosted 2 days agoPosition OverviewWe are seeking a motivated, creative, and hands-on Real Estate Agent Assistant to support a busy real estate professional. This role is ideal for someone who enjoys a mix of marketing, creativity,...