Skip to main content
Prompt Learning covers GEPA and MIPRO workflows—fast iterations that optimize prompts without re-training model weights. The CLI enforces end-to-end reproducibility by reading everything from your config and environment.

​
When to Use Prompt Learning

  • You want rapid improvements to system prompts or candidate instructions with automated scoring.
  • You already have a task app (the same environment used for RL) and just need to optimize textual guidance.
  • You prefer an online process that streams progress and delivers a Markdown summary when complete.

​
Requirements

  • Config: TOML specifying prompt_learning.algorithm (gepa or mipro), evaluation settings, datasets or candidate pools, and any scoring parameters.
  • Secrets: Both SYNTH_API_KEY and ENVIRONMENT_API_KEY are required. Store TASK_APP_URL alongside them so the CLI doesn’t have to prompt.
  • Task app: Must respond to check_task_app_health; fix outages before launching a run.
  • Experimental models: Gate access with --allow-experimental or --no-allow-experimental.

​
Launch a Run

uvx synth-ai train \
  --type prompt_learning \
  --config configs/prompt/gepa.toml \
  --env-file .env.prompt
  1. Config lock-in: Prompt learning ignores --task-url overrides; everything comes from the TOML to avoid mismatched environments.
  2. Env resolution: Same .env picker used in other flows, but it insists on both Synth and environment keys.
  3. Health check: Uses check_task_app_health exactly like RL; failures stop the flow before job creation.
  4. Payload preview & job creation: Calls POST {backend}/prompt-learning/online/jobs and prints the truncated response for auditing.

​
Streaming Options

  • --stream-format cli shows summarized events (prompt.learning.progress, prompt.learning.final.results, etc.) while hiding noisy substrings.
  • --stream-format chart draws gepa.transformation.mean_score over time—helpful for spotting convergence.
  • Adjust --poll-timeout / --poll-interval as needed; jobs typically finish faster than RL/SFT because they don’t retrain weights.

​
Results You Receive

  • Live events describing discovered candidates, GEPA phases, or MIPRO evaluations.
  • Final status JSON with success/failure plus backend metadata.
  • Local Markdown report saved beside the referenced config. It contains:
    • Best prompt and score, plus baseline metrics.
    • Attempted vs optimized candidates.
    • MIPRO Top-K table (rank, train/test scores, lifts, instruction snippets).
    • Extracted text replacements for quick copy/paste into future configs.

​
Tips & Troubleshooting

  • Because prompt learning reads task URLs directly from the config, double-check those values before running; the CLI will not incorporate --task-url.
  • If health checks fail, verify ENVIRONMENT_API_KEY and TASK_APP_URL in your .env; the CLI masks values but prints which keys were found.
  • Use --allow-experimental temporarily when testing unreleased models; remove it from automation to avoid surprises.