Which version do people prefer?

Put two replies, headlines or model answers side by side and see which one people would rather get, and why.

How it works for this decision

Your agent sends

{
  "kind": "judgment",
  "format": "compare",
  "question": "Which reply would you rather receive?",
  "rubric": "Warm and clear, never pushy.",
  "compare": {
    "a": "Sorry about the wait! Your refund went out today.",
    "b": "Your refund was processed on 11/11. Allow 5 to 7 business days."
  }
}

A verified person sees

Task$0.05

Which reply would you rather receive?

A: Sorry about the wait! Your refund went out today. B: Your refund was processed on 11/11. Allow 5 to 7 business days.

You get back

{
  "answer": "a",
  "agreement": 0.6,
  "answers": 5,
  "votes": {
    "a": 3,
    "b": 1,
    "same": 1
  },
  "contested": false,
  "reasons": [
    {
      "answer": "a",
      "reason": "It says sorry and gives the news first."
    },
    {
      "answer": "b",
      "reason": "B tells me when the money arrives."
    }
  ]
}

The result shown is an example. Try answering the task yourself.

Why a human check

Teams pick between two drafts by gut feel or by asking a model, and a model judging another model's writing shares its blind spots. Running a real survey for every choice is slow.

A panel of verified people, each judging on their own, gives you a preference and how close it was. When people genuinely split, Plures says so instead of pretending there's a winner, and a sample of their reasons tells you what to change.

Questions

Isn't this just a survey?

It's a survey that answers in minutes, through an API, from ID verified people, with a stable panel quality score behind every answer. Run it for every decision your agent makes, not once a quarter.

Does disagreeing hurt verifiers?

No. Judgment tasks have no right answer, so people are scored on consistency and attention instead. Honest minority views are what you want to hear.

How do I start?

Get your API key at plures.si. Start with your own team: the first 50 decisions each month are free. Add credit when you want ID verified people on the rest, live right away.

More use cases