Does the image follow the prompt?
Rate how well generated images match what was asked for, with people's reasons, for evals and training data.
How it works for this decision
Your agent sends
{
"kind": "judgment",
"format": "rate",
"question": "How well does this image match the prompt?",
"context": "Prompt: a red bicycle leaning on a blue door in the rain.",
"image_url": "https://example.com/bike.png",
"rubric": "Every object in the prompt, in the right colors."
}A verified person sees
How well does this image match the prompt?
Prompt: a red bicycle leaning on a blue door in the rain. The image is sent as image_url.
You get back
{
"answer": "4",
"agreement": 0.6,
"answers": 5,
"votes": {
"1": 0,
"2": 0,
"3": 1,
"4": 3,
"5": 1
},
"mean": 4,
"spread": 0.632,
"contested": false
}The result shown is an example. Try answering the task yourself.
Why a human check
Image and video models are judged by other models or by a few people on the team. Neither scales, and neither tells you what an ordinary person sees.
A panel rating each image from 1 to 5 against the prompt gives you an average, how much people disagreed, and short reasons: the human preference data labs and product teams use to make models better.
- Send kind "judgment" with format "rate", the prompt as context and the image as image_url.
- The result has mean and spread: a high spread means people saw it differently.
- Compare two models' images with format "compare".
Questions
Can I use the answers as training data?
You own what you send, and every result comes back to you through the API and your dashboard. If you plan to train on results at scale, email hello@plures.si and we'll set it up with you.
How do I start?
Get your API key at plures.si. Start with your own team: the first 50 decisions each month are free. Add credit when you want ID verified people on the rest, live right away.
More use cases
- Evals and AI qualityEvals: grade what your AI saysDoes the answer stick to what its sources say?YesPartlyNo3 of 3 verified people agreedHave people grade a sample of your assistant's answers against a rubric, and see quality drift before customers do.See how it works
- Evals and AI qualityHuman feedback for training and tuning modelsWhich answer is more helpful?Answer AAnswer BAbout the same3 of 3 verified people agreedAsk verified people which of two answers is better, and use their agreement as clean preference data.See how it works
- CustomersCatch upset customers your support agent misreadsIs this customer upset?YesNo3 of 3 verified people agreedHave people check tone and intent before an AI support agent closes, refunds or escalates a ticket.See how it works