Skip to main contentsynth_ai.data.rubrics
Rubric and Criterion data models.
These are INPUT/DEFINITION structures that users create to define evaluation criteria.
For OUTPUT/RESULT structures (scores after evaluation), see judgements.py.
Classes
CriterionExample
Example for a criterion showing expected scores.
Criterion
Single evaluation criterion within a rubric.
Flexible variant allowing weights > 1.0 and no normalization requirement.
Used by task apps for general reward computation.
Attributes:
id: Unique identifier for this criteriondescription: Human-readable description of what this criterion evaluatesweight: Relative weight for scoring (must be positive, default 1.0)required: Whether this criterion must be satisfied for success
Rubric
Rubric definition for evaluating task app outcomes.
Supports flexible aggregation and blending. Criteria weights do not need
to sum to 1.0, making this suitable for general task app usage.
Attributes:
version: Version string for this rubric definitiongoal_text: Optional human-readable description of the evaluation goalcriteria: List of Criterion objects defining evaluation criteriaaggregation: How to aggregate criterion scores (‘sum’, ‘weighted_sum’, ‘custom’, ‘inherit’)
