Skip to main content

​
What You’ll need

  1. A Synth account — create one with https://usesynth.ai/signup
  2. Modal account — create one at https://modal.com

​
Step 1: Set up the demo

uvx synth-ai demo
Select option 1, Math Single-Step (Modal deployment) 
Select a demo template:
  [1] Math Single-Step (Modal deployment) (math-modal)
      Packaged modal task app matching examples/rl math environment.
  [2] Crafter GRPO (local FastAPI) (crafter-local)
      Lightweight wrapper around examples/warming_up_to_rl/task_app/grpo_crafter for local experimentation.
  Enter choice [1-2] (default 1): 1
Once a demo is chosen, its necessary files will be saved to your current working directory 
Destination directory [/Users/jacob-roddy-beck/rl/math_demo]:
Demo template 'Math Single-Step (Modal deployment)' materialised at /Users/jacob-roddy-beck/rl/math_demo

​
Step 2: Fetch your Synth credentials

Automatically connect to your Synth account to fetch and locally store the credentials needed for this demo 
uvx synth-ai setup
These credentials are saved to your demo’s .env, which was generated via uvx synth-ai demo, for use in next steps 
Keys saved to: /Users/jacob-roddy-beck/rl/math_demo/.env

​
Step 3: Deploy task app to Modal

Deploy the math task app to Modal 
uvx synth-ai deploy
The deployment will package your task app and deploy it to Modal. Once complete, you’ll see the hosted URL: 
✓ Deployed math task app to Modal
URL: https://jacob-roddy-beck--hendrycks-math-task-app-demo-web.modal.run
Copy this URL and add it to your .env file: 
TASK_APP_BASE_URL=https://jacob-roddy-beck--hendrycks-math-task-app-demo-web.modal.run

​
Step 4: Train

uvx synth-ai train
Choose the RL config from your demo. This should default to your math RL config: 
Select a training config:
  1) [rl] /Users/jacob-roddy-beck/rl/math_demo/configs/rl_from_base_qwen17.toml (last used)
  ...
  0) Abort
  Enter choice [1]: 1
Once selected, your RL training will kick off

​
Step 5: View your run

Live status of your run will feed into your dashboard on https://usesynth.ai
⌘I