Skip to main content

​
synth_ai.sdk.graphs.verifier_schemas

Verifier API Contract Schemas These schemas define the expected structure for requests and responses to the verifier scoring endpoint at POST /api/graphs/verifiers/completions. Zero-shot verifier graphs use the same response format via POST /api/graphs/completions. This is the canonical contract that the backend MUST conform to.

​
Classes

​
EvidenceItem

Evidence item recorded during RLM evaluation via add_evidence tool.

​
CriterionScorePayload

Per-criterion score returned by the verifier.

​
ReviewPayload

Rubric review (event-level or outcome-level).

​
VerifierScoreResponse

Response body for POST /api/graphs/verifiers/completions. This is the canonical contract that verifier backends MUST return and is also used as the zero-shot verifier graph output. Methods:

​
aggregate_event_reward

aggregate_event_reward(self) -> Optional[float]
Aggregate all event totals into a single reward. Returns:
  • Sum of all event_totals, or None if empty

​
aggregate_outcome_reward

aggregate_outcome_reward(self) -> Optional[float]
Extract outcome reward from outcome_review. Returns:
  • outcome_review.total, or None if no outcome review

​
VerifierTaskApp

Task application metadata.

​
VerifierOptions

Verifier provider and configuration options.

​
VerifierTracePayload

Trace payload containing trajectory context.

​
VerifierScoreRequest

Request body for POST /api/graphs/verifiers/completions.

​
CalibrationExampleInput

Input schema for a calibration example (few-shot verifier). Validates that the example has a valid trace and matching rewards/objectives. Uses synth_ai.data.rewards.CalibrationExample dataclass for structure. Methods:

​
normalize_trace

normalize_trace(cls, data: Any) -> Any

​
validate_rewards_match_trace

validate_rewards_match_trace(self) -> CalibrationExampleInput
Validate that event rewards/objectives length matches trace events.

​
to_dataclass

to_dataclass(self) -> CalibrationExample
Convert to synth_ai.data.rewards.CalibrationExample dataclass.

​
GoldExampleInput

Input schema for a gold example (contrastive verifier). Validates that the example has required fields with correct types. Uses synth_ai.data.rewards.GoldExample dataclass for structure. Methods:

​
normalize_trace

normalize_trace(cls, data: Any) -> Any

​
to_dataclass

to_dataclass(self) -> GoldExample
Convert to synth_ai.data.rewards.GoldExample dataclass.