Does the image follow the prompt?

Rate how well generated images match what was asked for, with people's reasons, for evals and training data.

How it works for this decision

Your agent sends

{
  "kind": "judgment",
  "format": "rate",
  "question": "How well does this image match the prompt?",
  "context": "Prompt: a red bicycle leaning on a blue door in the rain.",
  "image_url": "https://example.com/bike.png",
  "rubric": "Every object in the prompt, in the right colors."
}

A verified person sees

Task$0.05

How well does this image match the prompt?

Prompt: a red bicycle leaning on a blue door in the rain. The image is sent as image_url.

You get back

{
  "answer": "4",
  "agreement": 0.6,
  "answers": 5,
  "votes": {
    "1": 0,
    "2": 0,
    "3": 1,
    "4": 3,
    "5": 1
  },
  "mean": 4,
  "spread": 0.632,
  "contested": false
}

The result shown is an example. Try answering the task yourself.

Why a human check

Image and video models are judged by other models or by a few people on the team. Neither scales, and neither tells you what an ordinary person sees.

A panel rating each image from 1 to 5 against the prompt gives you an average, how much people disagreed, and short reasons: the human preference data labs and product teams use to make models better.

Questions

Can I use the answers as training data?

You own what you send, and every result comes back to you through the API and your dashboard. If you plan to train on results at scale, email hello@plures.si and we'll set it up with you.

How do I start?

Get your API key at plures.si. Start with your own team: the first 50 decisions each month are free. Add credit when you want ID verified people on the rest, live right away.

More use cases