Skip to main content
This walkthrough shows how to run GEPA Online on the Banking77 intent classification task. In online mode, you drive rollouts locally while the backend:
  • proposes new instruction candidates as rewards arrive
  • selects candidates via a proxy URL (prompt substitution)
  • tracks per-candidate reward statistics and a live best prompt

​
Prerequisites

  • Python 3.11+
  • uv package manager
  • SYNTH_API_KEY set in your environment
  • Access to a Synth backend (default is production)

​
Run The Demo Script

From the synth-ai repo: 
cd path/to/synth-ai

export SYNTH_API_KEY="sk_live_your_key"
export SYNTH_URL="https://api.usesynth.ai"

uv run python demos/gepa_banking77/run_online_demo.py \
  --rollouts 60 \
  --train-size 50 \
  --val-size 10 \
  --min-proposal-rollouts 10

​
Local Dev (backend on your laptop)

If you are running the backend locally (compose infra + host processes), point the demo at localhost: 
export SYNTH_API_KEY="sk_dev_00000000000000000000000000000001"
export SYNTH_URL="http://localhost:8000"

uv run python demos/gepa_banking77/run_online_demo.py \
  --backend-url http://localhost:8000 \
  --rollouts 60 \
  --min-proposal-rollouts 10
If the demo only ever reports baseline as the candidate, your proposer is not running. For local dev, ensure the Rust backend has DEV_OPENAI_API_KEY set (or override the proposer model to one you have access to): 
export GEPA_ONLINE_PROPOSER_MODEL="gpt-4.1-mini"

​
What Happens

  1. The demo creates an online GEPA session via GepaOnlineSession.
  2. The backend returns a proxy URL (e.g. /api/gepa/v1/{session_id}/chat/completions).
  3. The demo calls the proxy URL for each LLM call.
  4. The proxy injects the currently-selected candidate instruction into the request (system message).
  5. The demo computes a reward (correct intent label or not) and reports it back to the session.
  6. Once enough rewards arrive, the backend proposes new instruction candidates and the proxy starts serving them.

​
See Also