​
Overview

Synth-AI supports any Hugging Face-compatible causal LM deployed via vLLM, plus OpenAI-compatible endpoints. Below are common choices validated in examples.

​
Validated examples

  • Qwen 0.6B / 1.8B (HF, via vLLM)
  • Llama 3.x (selected sizes)
  • Mistral 7B

​
Considerations

  • Ensure tokenizer path aligns with weights if using custom merges.
  • For RL, configure max_tokens, max_model_len, and batch sizes consistent with GPU memory.