Human feedback for training and tuning models

Ask verified people which of two answers is better, and use their agreement as clean preference data.

How it works for this decision

Your agent sends

{
  "question": "Which answer is more helpful?",
  "context": "Question: How do I reset my router? Answer A: Unplug it for 30 seconds, plug it back in, and wait two minutes for the lights to settle. Answer B: Routers can be reset in several ways depending on the model and manufacturer.",
  "options": [
    "Answer A",
    "Answer B",
    "About the same"
  ],
  "verifiers": 3
}

A verified person sees

Task$0.05

Which answer is more helpful?

Question: How do I reset my router? Answer A: Unplug it for 30 seconds, plug it back in, and wait two minutes for the lights to settle. Answer B: Routers can be reset in several ways depending on the model and manufacturer.

You get back

{
  "answer": "Answer A",
  "agreement": 1,
  "answers": 3,
  "votes": {
    "Answer A": 3,
    "Answer B": 0,
    "About the same": 0
  }
}

The result shown is an example. Try answering the task yourself.

Why a human check

Preference data and eval labels from anonymous crowds are noisy: rushed clicks, bots and people who never read the answer. Training on that noise teaches a model the wrong lesson.

Every Plures verifier passed an ID check and is scored on hidden tasks with known answers, and each answer comes with how strongly people agreed. Keep the pairs they agree on, look again at the split ones.

Shuffle which answer is A and which is B, so position doesn't sway the vote.
Use agreement as a label weight: train on clear wins, review the ties.
Put the model name and prompt id in metadata; verifiers never see it.

Questions

How many pairs can I send?

As many as you need. Each pair is its own task, and results arrive by webhook as people answer.

Can people judge specialist answers?

Tasks that need a skill only go to verifiers who qualify for it. For your own experts, route the task to your team.

How do I start?

Get your API key at plures.si. Start with your own team: the first 50 decisions each month are free. Add credit when you want ID verified people on the rest, live right away.