Evaluation Principles

The purpose of these principles is to define what counts as meaningful assessment in a non-anthropocentric setting. “Intelligence” here is not an honorific label and not a proxy for human IQ; it is a property of architectures that can sustain adaptive organization across conditions.

Diagnostics prioritize architecture-level properties: how a system represents constraints, maintains coherence under perturbation, and changes its own effective rules within declared boundaries. In MAS terms, this corresponds to separating state dynamics from architecture dynamics and evaluating admissible transformations rather than only their outputs.

Every evaluation is performed within an explicit descriptive layer: a model/language that determines what is observable and what counts as evidence. We separate (i) the system under study from (ii) the descriptive instruments used to represent it. This reduces “metric drift” where changes in description are mistaken for changes in the system.

Metrics are treated as diagnostics, not slogans. A valid metric must be anchored in observable traces, tied to a clear architectural interpretation, and robust to reasonable changes in tasks or data sources. A metric that cannot be stress-tested is not a metric; it is a mood.

Where possible, we prefer measures that remain stable under benign transformations (e.g., paraphrase, domain shift, sampling changes) while remaining sensitive to genuine architectural differences. Robustness checks are part of the evaluation logic, not an optional add-on.

Results should be communicated with explicit uncertainty awareness: assumptions, limits, scope of validity, and known failure modes. When evidence is weak or layer-dependent, the correct output is a bounded claim, not a confident narrative.