Skip to main content
Prompt optimization jobs consume TOML configs that describe the algorithm, initial prompt template, training/validation seeds, and optimization parameters. The CLI converts this into a job payload and submits it to Synth AI.

​
1. Create the Config TOML

Create a TOML file that follows the schema documented in this guide (see Key Parameters section below).

​
Example: Banking77 GEPA Configuration

[prompt_learning]
algorithm = "gepa"
task_app_url = "https://your-task-app.usesynth.ai"
task_app_id = "banking77"

# Training seeds (30 seeds from train pool)
evaluation_seeds = [50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79]

# Validation seeds (50 seeds from validation pool - not in training)
validation_seeds = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49]

[prompt_learning.initial_prompt]
messages = [
  { role = "system", content = "You are a banking intent classification assistant." },
  { role = "user", pattern = "Customer Query: {query}\n\nClassify this query into one of 77 banking intents." }
]

[prompt_learning.gepa]
initial_population_size = 20    # Starting population of prompts
num_generations = 15            # Number of evolutionary cycles
mutation_rate = 0.3             # Probability of mutation
crossover_rate = 0.5            # Probability of crossover
rollout_budget = 1000           # Total rollouts across all generations
max_concurrent_rollouts = 20    # Parallel rollout limit
pareto_set_size = 20           # Size of Pareto front

​
2. Launch the Job

import os
from synth_ai.sdk import PolicyOptimizationJob

async def run_training():
    client = PromptLearningClient(api_key=os.environ["SYNTH_API_KEY"])

    # Create job from TOML config
    job = await client.create_job_from_toml("configs/prompt_learning/banking77_gepa.toml")
    print(f"Created job: {job['id']}")

    # Start the job
    await client.start_job(job["id"])

    # Poll until completion
    result = await client.poll_until_terminal(job["id"])
    print(f"Best score: {result['best_score']}")
    return result
Steps performed by the SDK:
  1. Validate the TOML (PromptLearningConfig) and ensure the algorithm is selected (gepa).
  2. Verify the task app via health check using ENVIRONMENT_API_KEY.
  3. Create and start the job; poll_until_terminal() waits until the job reaches a terminal state.
Reminder: Load environment variables from .env using python-dotenv or your preferred method. Get API keys from the dashboard or run synth-ai setup.

​
Algorithm Selection

​
GEPA (Genetic Evolution for Prompt Optimization)

Best for: Classification tasks, multi-hop QA, instruction following
  • Uses evolutionary operations (mutation, crossover, selection)
  • Maintains a Pareto front balancing accuracy, token count, and task-specific metrics
  • Typically achieves 60-75% → 85-90%+ accuracy over 15 generations

​
Key Parameters

ParameterDescriptionTypical Range
initial_population_sizeStarting number of prompt variants10-50
num_generationsEvolutionary cycles to run5-30
mutation_rateProbability of mutating a prompt0.1-0.5
crossover_rateProbability of combining two prompts0.3-0.7
rollout_budgetTotal task evaluations allowed200-2000
max_concurrent_rolloutsParallel rollout limit10-50

​
Optional: Verifier Configuration

You can enable LLM-based verifying to evaluate subjective quality aspects. Add a [prompt_learning.verifier] section to your config: 
[prompt_learning.verifier]
enabled = true
reward_source = "fused"            # Combine environment and verifier rewards
synth_verifier_id = "banking-v1"   # Registered Verifier Graph or Rubric
backend_model = "gpt-4o-mini"
See the Verifiers Schema Reference for more details.