Skip to content

Conformance

Conformance suite map: checked corpus, standards-facing claims, and evidence surface for language/runtime compatibility.

For day-to-day language questions, start with the specification index. Release or compatibility claims belong next to the evidence surface that backs them.

Public claims should point at a suite, corpus, or reference page instead of marketing text.

Suite Overview

This directory defines the deterministic conformance suite contract for trust-platform and external runtime/tool comparisons.

The original Deliverable 1 MVP remains available as the v1 contract. The expanded v2 contract is the current flagship suite used by CI for public language/runtime proof.

Contract Versions

v1 Frozen Categories

The v1 contract is frozen to the six Deliverable 1 categories:

  1. timers
  2. edges
  3. scan_cycle
  4. init_reset
  5. arithmetic
  6. memory_map

Suites containing only these categories continue to emit trust-conformance-v1 summaries compatible with conformance/schemas/summary-v1.schema.json.

v2 Expanded Categories

The v2 suite extends v1 with:

  1. strings
  2. arrays
  3. structs
  4. enums
  5. nested_values
  6. oop_dispatch
  7. references
  8. retain_matrix
  9. scheduler
  10. comms_determinism

Suites containing any expanded category emit trust-conformance-v2 summaries compatible with conformance/schemas/summary-v2.schema.json.

Repository Layout

conformance/
  README.md
  contract.md
  naming.md
  schemas/
    summary-v1.schema.json
    summary-v2.schema.json
  cases/
    <category>/
      <case_id>/
        program.st
        manifest.toml
  expected/
    <category>/
      <case_id>.json
  reports/
    .gitkeep

Generated reports are CI/local artifacts and are not committed under conformance/reports/.

Determinism Contract

  • Case execution order is lexicographic by case_id.
  • Inputs and expected outputs are versioned in-repo.
  • A case only passes when observed results match expected artifacts exactly.
  • Output summaries must comply with the schema matching the emitted version/profile.
  • Communication determinism cases use simulated or loopback state transitions only. They must not depend on live sockets, brokers, PLCs, or fieldbus hardware.

Documents

  • Contract: conformance/contract.md
  • Naming rules: conformance/naming.md
  • Summary schemas: conformance/schemas/summary-v1.schema.json, conformance/schemas/summary-v2.schema.json
  • Failure taxonomy: conformance/failure-taxonomy.md
  • External run guide: conformance/external-run-guide.md
  • Known gaps: conformance/known-gaps.md
  • External submission process: conformance/submissions.md

Running The Suite

Generate or refresh expected artifacts:

trust-runtime conformance --suite-root conformance --update-expected

Run verification against versioned expected artifacts:

trust-runtime conformance --suite-root conformance

Optional output override:

trust-runtime conformance --suite-root conformance --output target/conformance/local-summary.json

Runner exits non-zero when any case is failed or error.

CI gate uses repeated runs and normalized summary comparison to verify deterministic ordering/status behavior.

Public Proof Surface

The current CI conformance gate runs the suite twice, normalizes timestamps and durations, diffs the summaries, validates the summary contract, and uploads both machine-readable JSON and human-readable Markdown reports as CI artifacts.

The committed conformance/expected/ artifacts are the runtime baseline. Generated files under conformance/reports/ are not part of the public docs source; use CI artifacts for run reports.