Skip to main content

​
Synth AI Graphs

Graphs are Synth AI’s abstraction for multi-node LLM graphs. Like task apps, graphs are first-class artifacts you can train, download, and serve in production.

​
What is a Graph?

A Synth AI graph is a directed graph of LLM calls and transformations. Each node can:
  • Call an LLM with a specific prompt template
  • Transform data between nodes
  • Branch conditionally based on intermediate results
  • Aggregate outputs from multiple paths
Unlike single prompts, graphs can express complex reasoning patterns: chain-of-thought, retrieval-augmented generation, self-consistency, and more.

​
Zero-Shot vs Optimized Graphs

Synth AI supports two ways to run graphs:
  1. Zero-Shot Graphs: Built-in, expert-designed architectures (Single, MapReduce, RLM) that you can use immediately with your own prompts or rubrics.
  2. Optimized Graphs: Custom graphs trained via Graph Evolve that have evolved their internal structure and prompts based on your specific dataset.

​
Graph Modes

Regardless of whether a graph is Zero-Shot or Optimized, it typically operates in one of two modes:
ModeInputOutputPurpose
Policyinput (JSON)output (JSON/Text)General inference, QA, and reasoning.
Verifiertrace (V3)score, reasoningVerifying quality and assigning rewards.

​
The RLM Architecture

RLM (Recursive Language Model) is Synth AI’s most powerful architecture, designed for complex reasoning loops and massive context windows (1M+ tokens). Unlike standard linear chains, RLM can:
  • Recursively decompose complex problems into smaller sub-tasks.
  • Audit and refine its own intermediate reasoning steps.
  • Scale with context, making it the primary choice for evaluating long agent trajectories or large documents.
See RLM for more details.

​
Graph Shapes & Scalability

Synth AI graphs support three distinct architectures (shapes) to handle different scales of input:
ShapePurposeContext Limit
singleSimple one-pass reasoning128k tokens
mapreduceSummarization/Evaluation of long logs500k tokens
rlmHigh-complexity tool-use and long-context reasoning1M+ tokens

​
Verifier Graphs

Verifier graphs evaluate execution traces and produce structured scores. At inference time, they take:
  1. A V3 trace - The execution trace from synth-ai tracing
  2. A rubric - Evaluation criteria defining what to score (optional if built into graph prompts)
{trace, rubric} → [Verifier Graph] → {score, event_rewards, reasoning}
Key use case: Training custom verifiers. Instead of using expensive frontier models (GPT-4, Claude) to evaluate outputs, you can train a verifier graph that:
  1. Matches human evaluation quality
  2. Runs on cheaper models (GPT-4o-mini, Groq)
  3. Provides consistent, calibrated scores
  4. Returns structured rewards (event-level and outcome-level)

​
Using Verifiers in GEPA

Once you have a registered Verifier Graph, you can use it to score trajectories during GEPA prompt optimization by setting the synth_verifier_id in your config: 
[prompt_learning.verifier]
enabled = true
reward_source = "verifier"
synth_verifier_id = "my-verifier-name"

​
Creating Graphs

Graphs are created through optimization. You provide:
  1. A dataset - Examples of inputs and expected outputs
  2. Configuration - Graph type, structure constraints, models to use
  3. A budget - How much optimization to run
Synth AI’s Graph Evolve algorithm then evolves an optimal graph structure and prompts. The simplest way to create graphs is through the Graph Evolve API: 
from synth_ai.sdk import GraphOptimizationJob

job = GraphOptimizationJob.from_dataset(
    "my_tasks.json",
    policy_model="gpt-4o-mini",
    rollout_budget=200,
)
job.submit()
result = job.stream_until_complete()
See Graphs for the full Graph Evolve documentation.