Safety measures, safety mechanisms, and error detection
STIG uses two complementary safety layers of protection:
tests, which exercise behaviour before a change is accepted (unit tests, integration tests, functional/verification tests);
runtime checks, which reject invalid inputs, impossible intermediate states, and untrustworthy generated artefacts while a real project is being analysed (postcondition checks, precondition checks, invariant checks).
extensive automated CI checks
Rough estimate mid 2026-07:
500 tests (i.e. during development). From them, unit tests alone bring 33% code coverage.
100 safety mechanisms cheks (i.e. at runtime)
Test layers
The current source-set ownership, suite boundaries, focused commands, and aggregate Gradle wiring are described in the testing overview. It deliberately avoids a static case count because Kotest suites and the native analyzer matrix expand dynamically. The C++ selector sanity source is also identified there as present but not currently wired into an executable test task.
Runtime checks
The validation hooks are not all one class hierarchy. They are deliberately placed where the task has the right context:
Kind | Current count | Examples |
|---|---|---|
Named validator objects | 2 |
|
| 57 | Analyzer JSON validation, generated-doc link checks, bitcode/debug-info checks, interface-point checks. |
Typed precondition problem sites | 10 | Missing inputs, unsupported analysis modes, missing symbols before analyzer launch. |
Typed postcondition problem sites | 27 | Missing generated outputs, analyzer JSON inconsistencies, symbol and diagram output violations. |
There are also direct text-message precondition/postcondition checks that do not yet construct typed StigProblem objects: 24 precondition-message sites and 42 postcondition-message sites. They are still runtime checks, but the long-term direction is to route continuable problems through the typed problem reporter so they can be collected in generated reports.
Placement rule
New checks should follow the validator placement rule:
put a check in the producer as a postcondition when the producer has enough context to verify its own output;
put a check in the first consumer that sees all required inputs when the invariant crosses parallel producers or external environment facts;
avoid duplicating the same invariant in two places unless the two checks have distinct diagnostic value.
This keeps failures close to the layer that introduced the bad state, while still allowing cross-task consistency checks where no single producer can know the whole invariant.