Rules

Rules are project instructions that adjust how specific AI outputs behave. A rule contains an instruction, an enabled state, and one or more outputs where it applies:

• Criterion scoring & comments — criterion scores, mistakes, and criterion-level explanations.
• Topic & general comment — topic classification and the overall evaluation comment.
• Attribute extraction — categorical, numerical, time, and day-time attributes.
• Flow evaluation — flow classification and step validation.
• QA dashboard notes — AI-generated internal manager notes.

Open Rules from the project navigation, select New rule, write the instruction, and select every applicable output. Keep each rule self-contained: the prompt already identifies the selected output, so write the behavior itself rather than phrases such as “when generating a dashboard note.”

Rules are applied in the order shown on the page. Use the up and down controls to organize them. Criterion rules also apply in Playground, Benchmarks, and criterion previews; topic and general-comment rules apply in Playground, Benchmarks, and general-comment regeneration. Attribute rules apply to normal and post-evaluation extraction, including attribute reapply; flow rules apply whenever flows are evaluated; QA dashboard-note rules apply only to note generation.

Changing a rule does not alter historical evaluations. The new version is used the next time an applicable output is generated. Project Description remains shared context supplied across prompts, so output-specific behavior belongs in Rules.

Writing useful rules

• State one coherent behavior per rule and select only the outputs that need it.
• Describe observable expectations and required format precisely.
• Avoid conflicting instructions; no rule has priority over another.
• Test evaluation rules in Playground and calibrate them with Benchmarks before relying on them at scale.