Skip to main content
Authenticated routes require an X-API-Key header set to your ENVIRONMENT_API_KEY (run uvx synth-ai setup to provision credentials). SFT task apps exist primarily to generate high-quality training data. Unlike RL, the training backend does not call your task app directly; instead, you use the app to gather rollouts/traces, export them to JSONL, and feed those files to synth-ai train --type sft. The app still exposes the standard HTTP interface so tools like uvx synth-ai eval, uvx synth-ai filter, and tracing_v3 know how to talk to it. Supervised data capture
  • Serialize conversations in the schema enforced by synth_ai.learning.sft.data (validate_training_jsonl, validate_jsonl_or_raise). Each JSONL line should resemble: 
    {
  "messages": [
    {
      "role": "user",
      "content": "...",            // str | list | dict | null
      "tool_calls": [],
      "tool_call_id": null,
      "name": null,
      "reasoning": null,
      "raw_content": null,
      "extra": {}
    }
  ],
  "tools": [],
  "tool_choice": null,
  "metadata": {},
  "extra": {}
}
  • Honour the tracing hooks used by synth_ai.task.tracing_utils:
    • TASKAPP_TRACING_ENABLED=1 turns tracing on.
    • TASKAPP_SFT_OUTPUT_DIR (or SFT_OUTPUT_DIR) tells resolve_sft_output_dir()/unique_sft_path() where to write JSONL batches.
    • TURSO_LOCAL_DB_URL, LIBSQL_URL, SYNTH_TRACES_DB, and SQLD_DB_PATH steer the tracing_v3 database so downstream tools can read rollouts.
  • Provide [filter] knobs compatible with synth_ai.task.config.FilterConfig (splits, score thresholds, model filters, limits) so uvx synth-ai filter can export curated JSONL.
Training payload pipeline
  • Ship SFT TOML configs that build_sft_payload accepts (dataset paths, model allowlist, hyperparameters, validation strategy).
  • After filtering, the SDK flow is FtClient.upload_training_file() → FtClient.create_sft_job() → FtClient.start_job().
Task app metadata expectations
  • Expose a TaskAppConfig whose base_task_info, describe_taskset(), and provide_task_instances() describe the dataset (task descriptor, environment id, dataset metadata, rubric, inference defaults, limits). These values surface through /info and /task_info.
  • rollout() responses must satisfy synth_ai.task.validators.validate_rollout_response_for_rl: pipeline_metadata.inference_url and every trajectory.steps[*].info.meta.inference_url need a ?cid=... token for trace correlation.
  • Hosting on Modal? Publish a modal.App script that imports your task app and call it with uvx synth-ai deploy --runtime modal (same requirements as RL deployments).
FastAPI endpoints:
  • /
    • Method: GET
    • Returns:
    { "status": "ok", "service": "task-app-id" }
  • /health
    • Method: GET
    • Returns:
    {
    "healthy": bool,
    "auth": {
        "required": bool,
        "expected_prefix": str
    }
}
  • /info
    • Method: GET
    • Returns:
    {
    "service": {
        "task": { "id": "...", "name": "...", "description": "...", "version": "..." },
        "version": "..."  // mirrors TaskDescriptor.version
    },
    "dataset": { "id": "...", "name": "...", "version": "...", "splits": ["train"], "default_split": "train" },
    "rubrics": {
        "outcome": { ... },   // optional RubricBundle sections
        "events": { ... }
    },
    "inference": { "model": "...", "inference_url": "..." },
    "limits": { "max_turns": 32, "max_response_tokens": 2048, "timeout_seconds": 60 }
}
  • /task_info
    • Method: GET
    • Query params: seed (int, repeatable) or seeds (array[int])
    • No seed returns:
    {
    "taskset": { ... }  // whatever cfg.describe_taskset() returns (split metadata, cardinality, etc.)
}
    • With seed(s) returns:
      • A single TaskInfo object (or a list of them) describing the seed-specific task (fields: task, environment, dataset, rubric, inference, limits, task_metadata).
  • /rollout
    • Method: POST
    • Body: RolloutRequest
    • Returns RolloutResponse:
    {
    "run_id": "...",
    "trajectories": [
        {
            "env_id": "...",
            "policy_id": "...",
            "steps": [
                {
                    "obs": {},
                    "tool_calls": [],
                    "reward": null,
                    "done": false,
                    "truncated": null,
                    "info": {
                        "meta": {
                            "inference_url": "https://.../?cid=trace_123"
                        }
                    }
                }
            ],
            "final": {},
            "length": 12,
            "inference_url": "https://.../?cid=trace_123",
            "decision_samples": null
        }
    ],
    "branches": {},
    "metrics": {
        "episode_returns": [1.0],
        "mean_return": 1.0,
        "num_steps": 12,
        "num_episodes": 1,
        "outcome_score": 0.9,
        "events_score": 0.8,
        "details": {}
    },
    "aborted": false,
    "ops_executed": 120,
    "trace_correlation_id": "trace_123",
    "trace": { "...": "SessionTrace payload" },
    "pipeline_metadata": {
        "inference_url": "https://.../?cid=trace_123"
    }
}