{{ site.title }}

{{ currentLang.label }}

English

{{ currentLang.label }}

English

{{ theme.darkModeSwitchLabel || 'Appearance' }}

Testing, Evaluation, and Quality Gates ​

Status: published 0.3.0 stable release. Supported platforms: Windows and Linux. macOS is not yet officially supported.

Purpose ​

Separate runtime outcome, metrics, business evaluation, and release readiness while preventing failures from masquerading as passes.

Public interfaces ​

  • checkProject
  • runEvaluationSuite
  • EvaluationGrader and the seven concrete grader types
  • RunOutcome
  • EvaluationReport
  • ReleaseReadiness
  • defineExperiment, selectExperimentArm, and runExperiment
  • ExperimentRecord

Errors and boundaries ​

  • Security gates cannot be overridden
  • Non-security gates require allowOverride plus an explicit reason and append a record to .coremind/quality-overrides.jsonl
  • An audit-write failure rejects the override
  • Strict scenarios run at least three times
  • schemaVersion 2 requires at least one outcome grader and allows at most 20 graders per scenario
  • Command, file, and diff evaluation enforce workspace, output, and timeout limits; pre-existing dirty-worktree content is preserved by default
  • Arm assignment is deterministic from experiment id, version, seed, and input fingerprint; it is reproducible and is not presented as cryptographic randomness
  • Every experiment record binds version, environment, input fingerprint, arm, terminal outcome, complete trace, and grader results

CoreMind supplies mechanisms, quality guardrails, and development guidance. Users or business owners retain control of goals, rules, data fields, approval ownership, and final acceptance.

Source, tests, and examples ​

{{ theme.lastUpdated?.text || theme.lastUpdatedText || 'Last updated' }}:

Released under the MIT License.

Copyright © CoreMind Contributors