Skip to main content
Mythos

πŸ“Eval-Driven Loops are an agent workflow in which output is repeatedly checked against an automated evaluation, such as tests, scores, or a grading model, and revised until it passes.

The loop has three parts: an agent produces a candidate, an evaluator measures it, and the result feeds back as guidance for the next attempt. Evaluators range from unit tests and type checkers to rubric-based scoring by a second model. The loop ends when the evaluation passes or a retry limit is reached.

The approach replaces subjective judgment with a defined bar. Agents tend to report success optimistically, and an external check gives both the agent and the user an objective signal. Writing the evaluation first, as in test-driven development, also forces the goal to be stated precisely before work begins.

Loop quality depends on evaluation quality. A weak test suite lets wrong output pass, and an agent optimizing against a narrow metric can satisfy the check without meeting the intent. πŸ“AI Agent Evaluation covers how evaluations are designed, and the same mechanism underlies self-healing tests and self-improving loops.

Contexts

Created with πŸ’œ by One Inc | Copyright 2026