Skip to main content

​
Graph Inference

After training a graph with Graph Evolve, or using a built-in Zero-Shot graph, you can run production inference through the Synth AI API. The graph executes server-side using optimized or default prompts and structure.

​
Unified Graph Inference

The /graphs/completions endpoint is the recommended way to run all types of graphs, including trained Policy/Verifier graphs and built-in Zero-Shot graphs.

​
API Endpoint

POST /api/graphs/completions

​
Request

{
  "job_id": "graph_gen_abc123", // Or built-in "zero_shot_*" ID
  "input": {
    "question": "What is the capital of France?"
  }
}
FieldTypeRequiredDescription
job_idstringYesGraph Evolve job ID or built-in zero_shot_* ID
inputobjectNoInput for Policy graphs
traceobjectNoV3 SessionTrace for Verifier graphs
rubricobjectNoEvaluation criteria for Verifier graphs
calibration_examplesarrayNoOptional few-shot examples
modelstringNoOverride the policy model for this call

​
Response

{
  "output": {
    "answer": "Paris"
  },
  "usage": {
    "prompt_tokens": 1024,
    "completion_tokens": 256,
    "total_tokens": 1280
  },
  "node_timing": {
    "node_1": 0.45,
    "node_2": 1.2
  },
  "metadata": {
    "latency_ms": 342,
    "model": "gpt-4o-mini",
    "snapshot_id": "snap_xyz789"
  }
}

​
Python SDK

​
Using GraphOptimizationJob

from synth_ai.sdk import GraphOptimizationJob

# After training completes or for built-in graphs
job = GraphOptimizationJob.from_job_id("graph_gen_abc123", api_key=api_key)

# Run inference
result = job.run_inference({
    "question": "What is the capital of France?"
})
print(result["output"])  # {"answer": "Paris"}

​
Verifier Graph Inference

Verifier graphs (custom verifiers) accept V3 traces and rubrics.

​
Request

{
  "job_id": "zero_shot_verifier_rubric_rlm",
  "trace": {
    "session_id": "trace_to_evaluate",
    "session_time_steps": [...]
  },
  "rubric": {
    "outcome": {
      "criteria": [
        {"name": "correctness", "description": "Is it correct?", "weight": 1.0}
      ]
    }
  }
}

​
Response

{
  "score": 0.92,
  "reasoning": "The agent correctly identified Paris...",
  "event_rewards": [
    {"event_id": 1, "value": 0.9, "annotation": "Good search query"}
  ],
  "outcome": {
    "total_reward": 0.92,
    "feedback": "Task completed correctly"
  }
}

​
Python SDK for Verifiers

from synth_ai.sdk import GraphOptimizationJob

# Load your trained or zero-shot verifier
verifier = GraphOptimizationJob.from_job_id("zero_shot_verifier_rubric_rlm", api_key=api_key)

# Evaluate a trace
result = verifier.run_inference(
    trace=trace_data,
    context={
        "rubric": {
            "outcome": {
                "criteria": [{"name": "accuracy", "weight": 1.0}]
            }
        }
    }
)

print(f"Score: {result['score']}")

​
Error Handling

The API returns standard HTTP status codes:
  • 413 Request Entity Too Large: Trace or input exceeds the limit for the selected graph shape.
  • 422 Unprocessable Entity: Invalid trace format or missing required fields.
  • 400 Bad Request: Missing rubric for rubric-based graphs.