Skip to main content
POST
/
api
/
graphs
/
verifiers
/
completions
from synth_ai.sdk.graphs import VerifierClient

client = VerifierClient()

result = await client.evaluate(
    job_id="zero_shot_verifier_rubric_single",
    trace=my_trace,
    rubric={
        "outcome": [
            {"id": "task_completion", "weight": 1.0, "description": "Task completed"}
        ]
    }
)

print(f"Score: {result.outcome_review.total}")

{
  "event_reviews": [],
  "outcome_review": {
    "total": 0.8,
    "reasoning": "Task completed successfully with minor issues."
  },
  "normalized_event_rewards": [],
  "metadata": {}
}
Evaluate a trace against rubric criteria and get structured reviews with aggregate rewards.
Verifiers are automatically called during eval jobs when you enable verifier evaluation in your config.
​
job_id
string
required
Verifier graph ID. Use zero_shot_verifier_rubric_single for basic evaluation or zero_shot_verifier_rubric_rlm for RLM-powered evaluation.
​
trace
object
required
V3 trace to evaluate.
​
rubric
object
required
Evaluation criteria.
​
options
object
Evaluation options.
from synth_ai.sdk.graphs import VerifierClient

client = VerifierClient()

result = await client.evaluate(
    job_id="zero_shot_verifier_rubric_single",
    trace=my_trace,
    rubric={
        "outcome": [
            {"id": "task_completion", "weight": 1.0, "description": "Task completed"}
        ]
    }
)

print(f"Score: {result.outcome_review.total}")

{
  "event_reviews": [],
  "outcome_review": {
    "total": 0.8,
    "reasoning": "Task completed successfully with minor issues."
  },
  "normalized_event_rewards": [],
  "metadata": {}
}

​
Example Request

Request
{
  "job_id": "zero_shot_verifier_rubric_single",
  "trace": {
    "schema_version": "3.0",
    "event_history": [],
    "markov_blanket_message_history": [],
    "metadata": {}
  },
  "rubric": {
    "event": [
      {
        "id": "legality",
        "weight": 1.0,
        "description": "Actions taken are legal and valid"
      }
    ],
    "outcome": [
      {
        "id": "task_completion",
        "weight": 1.0,
        "description": "The task was completed successfully"
      }
    ]
  },
  "options": {
    "event": true,
    "outcome": true,
    "provider": "openai",
    "model": "gpt-4o-mini"
  }
}

​
Built-in Verifier Graphs

Graph IDDescription
zero_shot_verifier_rubric_singleBasic rubric evaluation
zero_shot_verifier_rubric_rlmRLM-powered evaluation for long traces

​
Using with Eval Jobs

When verifier evaluation is enabled in an eval job:
  1. Traces are automatically hydrated from the interceptor
  2. Rubric is pulled from the container’s /info endpoint
  3. Container reward and verifier reward are fused into a final score