CI Test Runner (`labwired test`)

LabWired provides a CI-friendly runner mode driven by a YAML test script:

labwired test --script test.yaml

Every v1.0 script selects exactly one run shape: a single machine or an environment world. The two inputs forms are mutually exclusive.

Single-machine script

Use inputs.firmware (and optionally inputs.system) to run one machine. A per-machine system.yaml is the same system manifest consumed by the Playground; CI does not use a separate board-description format.

labwired test --firmware path/to/fw.elf --system system.yaml --script test.yaml

schema_version: "1.0"
inputs:
  firmware: "relative/or/absolute/path/to/fw.elf"
  chip: "stm32f103"
limits:
  max_steps: 100000
assertions:
  - uart_contains: "Hello"

inputs.chip names one of the chips bundled with the CLI, for firmware with nothing wired to it. This is the whole configuration — there is no manifest to write and no chip descriptor to copy into your repository, so your test tracks our silicon model instead of freezing a copy of it. Run labwired chips for the available names.

Firmware with external devices or board I/O uses inputs.system instead, pointing at a manifest:

inputs:
  firmware: "path/to/fw.elf"
  system: "path/to/system.yaml"

A manifest's chip: field takes the same two forms: a built-in name, or a path to your own descriptor for silicon we do not ship. Setting both inputs.chip and inputs.system is an error.

--firmware and --system are single-machine overrides only. They are not valid with an environment script.

Environment/world script

Use inputs.env to select an environment manifest. The manifest owns the world topology: every node supplies its id, the shared-with-Playground system.yaml, and its firmware ELF. Connectivity belongs only in explicit interconnects; it is never implied by CLI overrides.

# test.yaml
schema_version: "1.0"
inputs:
  env: "two-node-env.yaml"
limits:
  max_steps: 100000
  max_cycles: 123456     # optional
  max_uart_bytes: 4096   # optional
  wall_time_ms: 5000     # optional
  stop_when_assertions_pass: true          # optional; default false
  stop_when_assertions_pass_settle_steps: 1000  # optional; default 100000
  stop_when_assertions_pass_min_steps: 1000     # optional; default 0
assertions:
  - memory_value:
      node: alpha
      address: 0x20000000
      expected_value: 0
      size: 8
      mask: 0xff

# two-node-env.yaml
schema_version: "1.0"
name: "two-node-smoke"
nodes:
  - id: alpha
    system: "systems/alpha.yaml"
    firmware: "firmware/alpha.elf"
  - id: beta
    system: "systems/beta.yaml"
    firmware: "firmware/beta.elf"
interconnects:
  - type: can_bus
    nodes: [alpha, beta]
    config:
      peripheral: can1

An environment assertion must be a node-qualified memory_value assertion. v0.19 environment worlds are Cortex-M-only: each resolved node chip must resolve to arch: arm and explicitly declare core: cortex-m*. Its firmware must be an EM_ARM ELF with a valid Cortex-M Thumb reset vector at flash.base + reset_vector_offset: the initial stack pointer must land in RAM and the Thumb-bit reset handler must land in flash. A node outside that contract is rejected as a configuration error before a Cortex-M machine is constructed.

nodes[].config_overrides must be omitted. Every explicit occurrence, including {} and null, is rejected in environment schema 1.0.

Interconnect membership and config are validated before the world starts. Each config mapping is closed and type-checked:

uart_cross_link requires exactly two unique, known nodes. Its only optional keys are non-empty strings node_a_uart and node_b_uart (each defaults to uart2).
can_bus requires at least two unique, known nodes. Its only key is required non-empty string config.peripheral.
egress requires exactly one known node. Its only keys are optional non-empty strings uart (default usart2), transport (tcp, mqtt, or http), encoding (raw, ndjson-trace, or frames-json), required non-empty string url, MQTT-only required non-empty string topic, and positive integer buffer_max. topic is invalid for TCP and HTTP.

Environment scripts do not accept single-machine firmware/system overrides or topology-affecting CLI options.

stop_when_assertions_pass is opt-in for an environment world and has the same durable completion contract as a single-machine run. Once every node-qualified memory_value assertion passes at or after stop_when_assertions_pass_min_steps, it must remain true for stop_when_assertions_pass_settle_steps world rounds (default 100000). A regression restarts that window. A runtime failure or a same-round wall_time_ms, max_cycles, or max_uart_bytes limit wins over completion. max_steps remains the outer execution bound, so completion on its final allowed world round reports assertions_passed; otherwise the result has stop_reason: assertions_passed. The three fields are respectively a boolean and non-negative integers; explicit YAML null is rejected.

Exit Codes

Code	Meaning	Notes
`0`	Pass	All assertions passed, and any non-success stop reason was explicitly asserted (if applicable).
`1`	Assertion failure	Includes failed assertions, and hitting `wall_time_ms` / `max_uart_bytes` / `no_progress_steps` without asserting the matching `expected_stop_reason`.
`2`	Config/script error	Invalid YAML, unknown fields, unsupported schema_version, missing/invalid inputs/limits, or a safety guard (e.g. max-steps cap).
`3`	Simulation/runtime error	Runtime failure (e.g. `memory_violation`, `decode_error`) unless an `expected_stop_reason` assertion matches the stop reason.

Exit-code precedence: 1) Any failed assertion ⇒ 1 (even if a runtime error also occurred) 2) wall_time / max_uart_bytes / no_progress stop without matching expected_stop_reason ⇒ 1 3) Runtime error stop without matching expected_stop_reason ⇒ 3 4) Otherwise ⇒ 0

Script Schema (v1.0)

schema_version: "1.0"
inputs:
  firmware: "relative/or/absolute/path/to/fw.elf"
  system: "optional/path/to/system.yaml"
limits:
  max_steps: 100000
  max_cycles: 123456     # optional
  max_uart_bytes: 4096   # optional
  no_progress_steps: 500 # optional (PC unchanged for N steps)
  wall_time_ms: 5000   # optional
assertions:
  - uart_contains: "Hello"
  - uart_regex: "^Hello.*$"
  - rtt_contains: "Boot complete"
  - fidelity_clean: true
  - peripheral_log: {peripheral: i2c1, log: bus_trace, contains: "addr 0x48 W ack"}
  - expected_stop_reason: max_steps

Notes: - Unknown fields are rejected (script parse/validation returns exit code 2). - Relative inputs.firmware / inputs.system paths are resolved relative to the directory containing the script file (not the current working directory). - Relative inputs.env is resolved relative to the test script; each node's system and firmware paths are resolved relative to the environment manifest. - For a single-machine script, CLI flags override script inputs: - --firmware overrides inputs.firmware - --system overrides inputs.system - For an environment script, inputs.env is the only topology input; --firmware and --system are rejected. - CLI flags override script limits: - --max-steps overrides limits.max_steps - --max-cycles overrides limits.max_cycles - --max-uart-bytes overrides limits.max_uart_bytes - --detect-stuck (alias: --no-progress) overrides limits.no_progress_steps - --breakpoint <addr> (repeatable) stops the run when PC matches and sets stop_reason: halt.

rtt_contains matches only the dedicated SEGGER RTT stream (see SEGGER RTT logging); its presence enables RTT capture automatically. Environment/world scripts reject it at validation.

fidelity_clean: true passes only when the run hit no fidelity gap: no access to unmapped MMIO and no undecoded instruction. These are the unmapped_mmio and undecoded_instruction entries of the fidelity section in result.json. Approximations such as derived_device_time are not gaps. fidelity_clean: false passes only when the run hit at least one gap. The check reads the report when the run ends, before other assertions read memory. Use it in place of the LABWIRED_STRICT_FIDELITY environment variable, which aborts the process at the first gap with no result.json.

peripheral_log checks a named log that a peripheral model records during the run. A line matches when it contains contains; the assertion passes when at least min_count lines match (default 1). A model can store consecutive identical events as one line with a repeat count: the line ends in x{count} (for example seq 2 cmd 0x05 addr 0x00000000 size 1 x5000). Such a line counts as count matches, and contains matches the text before the x{count}. The check runs once, when the run ends. It does not hold a stop_when_assertions_pass run open.

assertions:
  - peripheral_log: {peripheral: usb1, log: host, contains: "device 1fc9:0135"}
  - peripheral_log: {peripheral: flexio2, log: wire, contains: "pin 2 ", min_count: 960}

peripheral is a peripheral name or the external_devices: id of an attached device (a device that records logs, such as the NAU88L21 codec):

assertions:
  - peripheral_log: {peripheral: codec, log: writes, contains: "write 0x001c = 0x000e"}

A peripheral, device or log name that does not exist is a config error (exit code 2), found before the run starts. The error lists the logs the peripheral has. A device has no bus_trace of its own: its traffic is in the bus_trace of the controller it hangs off. Logs today:

Log	Peripheral	One line per	Example line
`bus_trace`	every peripheral	bus-trace event with this peripheral's name	`addr 0x54 W nack`, `data 0x0f ack`, `mosi 0x9f miso 0xef`, `tx 0x41`
`host`	i.MX RT USB OTG (`imxrt_usb`)	fact or transfer the simulated USB host saw	`device 1fc9:0135 class 00/00/00 usb 0200 ep0 64`, `address 5`, `interface 3 alt 0 class 03/00/00 endpoints 2`, `endpoint 0x81 attr 0x03 max 64`, `string 2 "Product"`, `control 00 09 0001 0000 0000 ok`, `in 1 01 02`, `out 2 64`, `note ...`
`ip`	i.MX RT FlexSPI (`imxrt_flexspi`)	run of identical IP commands	`seq 1 cmd 0x9f addr 0x00000000 size 3`, `seq 2 cmd 0x05 addr 0x00000000 size 1 x5000`
`wire`	i.MX RT FlexIO (`imxrt_flexio`)	word shifted out on a pin	`pin 2 width 1 bits 8 beats 0xc0 cycle 1234`
`writes`	NAU88L21 codec (`nau88l21`, attached device)	register write	`write 0x001c = 0x000e`, `write 0x0000 = 0x0000 (reset)`
`reads`	NAU88L21 codec (`nau88l21`, attached device)	register word read	`read 0x0058 = 0x1a20`
`regs`	NAU88L21 codec (`nau88l21`, attached device)	register, value at the end of the run	`0x001c = 0x000e`
`state`	NAU88L21 codec (`nau88l21`, attached device)	decoded audio interface setting	`dai slave i2s 32-bit`, `adcout driven`, `enable dac_l dac_r adc_l adc_r`
`text`	multiplexed segment display (`segment-display-mux`, external device)	change of the text a human sees	`"P3C" at cycle 3200000000`
`frames`	multiplexed segment display (`segment-display-mux`, external device)	change of the visible segment masks, one per digit	`0x0629 0x0000 0x0000 at cycle 3200000000`
`frames`	multiplexed LED matrix (`led-matrix-mux`, external device)	change of the picture, 0–9 per pixel, one group per row	`09090 99999 99999 09990 00900 at cycle 3200000`

The bus trace is a ring of 4096 events shared by all buses. A long run can drop early events. When a bus_trace check fails, the message tells how many events the ring dropped. A device log is kept by the device model and is not in the ring. A model adds a log by overriding Peripheral::logs in labwired-core. An external device is named by its external_devices id and has no bus_trace log: a device on GPIO pads (a BusResidentDevice) adds a log by overriding BusResidentDevice::logs, an attached I2C device by overriding I2cDevice::logs. When a check fails, the message shows the last lines of the log.

The single-machine example above permits the documented single-machine assertions. Environment scripts are stricter: they require at least one node-qualified memory_value assertion and support max_steps, max_cycles, max_uart_bytes, wall_time_ms, stop_when_assertions_pass, stop_when_assertions_pass_settle_steps, and stop_when_assertions_pass_min_steps limits. They reject limits.no_progress_steps and single-machine-only controls such as --breakpoint.

Deprecated Legacy Schema (v1)

For backward compatibility, schema_version: 1 is still accepted, but is deprecated and will be removed in a future release.

Legacy shape:

schema_version: 1
firmware: "optional/path/to/fw.elf"   # optional (can be provided by --firmware)
system: "optional/path/to/system.yaml"
max_steps: 100000
wall_time_ms: 5000   # optional
assertions: []

Stop Reasons

expected_stop_reason supports: - max_steps - max_cycles - max_uart_bytes - no_progress - wall_time - assertions_passed - memory_violation - decode_error - halt - exception - config_error (runner failed before simulation started; e.g. script parse/validation error)

Semantics: - If the simulator hits wall_time_ms, the run is treated as an assertion failure (exit code 1) unless an expected_stop_reason assertion matches wall_time. - If the simulator hits max_uart_bytes or no_progress_steps, the run is treated as an assertion failure (exit code 1) unless an expected_stop_reason assertion matches (max_uart_bytes / no_progress). - If the simulator hits max_steps or max_cycles, the run is considered a normal stop (exit code 0) as long as assertions pass. - max_cycles, after_cycles triggers and a stimulus's reported at_cycle all count machine cycles (Machine::total_cycles): one per retired instruction on Cortex-M, RISC-V and Xtensa, and the core's datasheet cycles (1–4 per instruction) on the ATmega328P. A limit or trigger lands on its cycle or at the first instruction boundary after it. The top-level cycles field in result.json is a performance counter and may differ. - If opt-in assertion completion reaches its durable settling window, the run stops with assertions_passed and passes. - If the simulator hits a runtime error stop reason (e.g. memory_violation), the run is treated as a runtime error (exit code 3) unless an expected_stop_reason assertion matches the stop reason.

result.json uses: - result_schema_version: the result-union discriminator: "1.0" for a single machine or "1.0-environment" for an environment world - run_type: "environment": present only on the environment arm - stop_reason: the terminal reason the simulator stopped - stop_reason_details: which stop condition triggered (+ the limit/observed value when applicable) - limits: the resolved limits used for the run (after applying any CLI overrides) - status: one of pass, fail, error

Artifacts

Use --output-dir to write artifacts:

labwired test --script test.yaml --output-dir out/artifacts

Artifacts: - out/artifacts/result.json: machine-readable summary - out/artifacts/snapshot.json: a CPU snapshot for a single machine, or an environment snapshot with a nodes array for a world (also written for config errors) - out/artifacts/uart.log: captured UART TX bytes; world output is grouped by node - out/artifacts/rtt.log: captured SEGGER RTT bytes (empty unless RTT was enabled) - out/artifacts/junit.xml: JUnit XML report (one testcase for run + one per assertion)

Alternatively, you can write JUnit XML to a specific path:

labwired test --script test.yaml --junit out/junit.xml

Path Resolution Rules

--script: if relative, resolved relative to the current working directory.
Script-relative paths:
inputs.firmware, inputs.system (v1.0)
inputs.env (environment v1.0)
firmware, system (legacy v1) are resolved relative to the directory containing the script file.
Environment-manifest-relative paths: each nodes[].system and nodes[].firmware value is resolved relative to the environment manifest.
System manifest-relative paths: system.yaml may reference chip: ...; this chip path is resolved relative to the directory containing the system manifest file.
--output-dir / --junit: if relative, resolved relative to the current working directory.

`result.json` Contract

The runner writes result.json only when --output-dir is provided, including config/script errors that exit with code 2. Its top-level shape is a discriminated union. Consumers must branch on result_schema_version—and, for worlds, run_type—rather than assuming one firmware identity.

JSON Schema

{
  "$schema": "https://json-schema.org/draft/2020-12/schema",
  "title": "LabWired CI runner result.json",
  "$defs": {
    "common": {
      "type": "object",
      "required": [
        "status",
        "steps_executed",
        "cycles",
        "instructions",
        "stop_reason",
        "stop_reason_details",
        "limits",
        "assertions"
      ],
      "properties": {
        "status": { "enum": ["pass", "fail", "error"] },
        "steps_executed": { "type": "integer", "minimum": 0 },
        "cycles": { "type": "integer", "minimum": 0 },
        "instructions": { "type": "integer", "minimum": 0 },
        "stop_reason": {
          "enum": [
            "config_error",
            "max_steps",
            "max_cycles",
            "max_uart_bytes",
            "no_progress",
            "wall_time",
            "assertions_passed",
            "memory_violation",
            "decode_error",
            "halt",
            "exception"
          ]
        },
        "stop_reason_details": { "type": "object" },
        "limits": { "type": "object" },
        "message": { "type": ["string", "null"] },
        "assertions": { "type": "array" },
        "fidelity": { "type": "array" }
      }
    },
    "environment_node": {
      "type": "object",
      "required": ["id", "system", "firmware", "system_hash", "firmware_hash"],
      "properties": {
        "id": { "type": "string" },
        "system": { "type": "string" },
        "firmware": { "type": "string" },
        "system_hash": { "type": "string" },
        "firmware_hash": { "type": "string" }
      }
    }
  },
  "oneOf": [
    {
      "title": "single-machine result",
      "allOf": [
        { "$ref": "#/$defs/common" },
        {
          "type": "object",
          "required": ["result_schema_version", "firmware_hash", "config"],
          "properties": {
            "result_schema_version": { "const": "1.0" },
            "firmware_hash": {
              "type": "string",
              "description": "SHA-256 of the firmware ELF bytes."
            },
            "config": {
              "type": "object",
              "required": ["firmware", "system", "script"],
              "properties": {
                "firmware": { "type": "string" },
                "system": { "type": ["string", "null"] },
                "script": { "type": "string" }
              }
            }
          }
        }
      ]
    },
    {
      "title": "environment/world result",
      "allOf": [
        { "$ref": "#/$defs/common" },
        {
          "type": "object",
          "required": ["result_schema_version", "run_type", "config"],
          "not": { "required": ["firmware_hash"] },
          "properties": {
            "result_schema_version": { "const": "1.0-environment" },
            "run_type": { "const": "environment" },
            "config": {
              "type": "object",
              "required": ["script", "environment", "world_firmware_hash", "nodes"],
              "properties": {
                "script": { "type": "string" },
                "environment": { "type": "string" },
                "world_firmware_hash": { "type": "string" },
                "nodes": {
                  "type": "array",
                  "items": { "$ref": "#/$defs/environment_node" }
                }
              }
            }
          }
        }
      ]
    }
  ]
}

The single-machine arm has the top-level firmware_hash and a config.firmware/system/script provenance object. The environment arm has no top-level firmware_hash: it identifies the world with config.world_firmware_hash and records each node's id, system, firmware, system_hash, and firmware_hash. A world config error still uses the environment arm; its nodes list can be empty when the manifest could not be resolved. That includes explicit config_overrides (including {} and null), a chip without an explicit Cortex-M core, an ARM ELF without a valid Thumb reset vector, and unknown, mistyped, or invalid interconnect configuration; these produce status: "error" with stop_reason: "config_error" without changing the result-union arm.

CI release runners

Use the pinned v0.23.0 release runner in CI. It runs the same labwired test command described above and writes the same artifact contract.

GitHub Actions

Use the public Core action and pin the Core CLI with its version input. Its only inputs are required script, optional version (default v0.23.0), output-dir, and args:

- id: labwired
  name: Run LabWired tests
  uses: w1ne/labwired-core/.github/actions/labwired-test@64ed5d9723e2d9f5f4a851a81a3468025671d94e
  with:
    version: v0.23.0
    script: examples/ci/dummy-max-steps.yaml
    output-dir: out/artifacts
    args: --no-uart-stdout

- name: Link the automatic LabWired artifact
  if: always()
  run: echo "${{ steps.labwired.outputs.artifact-url }}" >> "$GITHUB_STEP_SUMMARY"

The Core action is an immutable action-source pin to 64ed5d9723e2d9f5f4a851a81a3468025671d94e; version: v0.23.0 independently pins the immutable Core CLI release. It downloads that public release archive with curl, creates output-dir/junit.xml plus Markdown and HTML reports, appends the Markdown report to the job summary, and always uploads the entire output directory—even after a failed test. Its status, summary-md, report-html, artifact-url, and exit-code outputs are available through the labwired step ID.

Docker and GitLab runners

The GHCR image has labwired as its entrypoint. For Docker, pass test directly after the pinned image name. Docker and the Action accept the same test YAML: use either a single-machine script or an inputs.env world script.

docker run --rm --user "$(id -u):$(id -g)" -v "$PWD:/workspace" -w /workspace \
  ghcr.io/w1ne/labwired:v0.23.0 \
  test --script examples/ci/dummy-max-steps.yaml \
       --output-dir out/artifacts \
       --no-uart-stdout

GitLab should clear that entrypoint and invoke labwired from its job shell:

image:
  name: ghcr.io/w1ne/labwired:v0.23.0
  entrypoint: [""]
script:
  - labwired test --script examples/ci/dummy-max-steps.yaml --output-dir out/artifacts --no-uart-stdout

Advanced source build

Build from source only when validating an unreleased LabWired commit. It is not the normal CI path:

cargo build --release -p labwired-cli
./target/release/labwired test --script examples/ci/dummy-max-steps.yaml --output-dir out/artifacts --no-uart-stdout

CI Test Runner (labwired test)