Validator placement rule
A consistent rule for where to put a postcondition / consistency check / validator across the stig pipeline. The pipeline has dozens of Gradle tasks that produce and consume each other's outputs, plus a few that run in parallel. Without a clear rule, validators land inconsistently — some at the producer, some at the consumer, some duplicated — and bugs slip through the gaps where each side assumes the other is checking.
The rule
If the check can be performed by the producer of the output — the task that generated or created the artefact being checked — then the check MUST be placed there as a postcondition.
Producer-side checks are preferred because:
They fire at the earliest possible point a wrong output exists.
The producer has full context (the inputs it was given + the output it produced) without needing to re-derive anything.
Failure messages can point directly at the offending input flag, DSL setting, or upstream task that triggered the bad output.
The check runs once per producer invocation, not once per downstream consumer.
Otherwise, the check is placed in the consumer.
"Otherwise" specifically means the producer can't verify the property — typically because it requires information from somewhere outside the producer's input/output graph. The most common case:
Cross-task consistency between non-sequenced producers. Two tasks
AandBrun in parallel (nodependsOnbetween them), and a consistency invariant relatesA.outputtoB.output. Neither A nor B can check it — each only sees its own half. The check goes in the first consumer that depends on both A AND B.External / environment cross-checks. E.g. verifying that the clang version a Gradle task assumes (
clangBinary = "clang-18") matches the LLVM version the analyzer binary was built against (LLVM_MAJORinpins.env). The producer task that compiles a.bcdoesn't itself know about the analyzer's pins; the downstream task that consumes both sides does.
Examples
Producer-side (Case 1)
StigBuildBitcodeTask.validateDebugInfoCoverage()scansmodule.llfor!llvm.dbg.cu+!dbgrefs right after the task's ownllvm-link+llvm-disfinish. The task produces the bitcode; the check verifies the produced bitcode has the property the user's flags claimed it would.The "Diagram-completeness" validator (
StigAnalysisP7GenerateDiagramsTask's end-of-run()postcondition): enumerates DSL-expected (analysis_type × depth × mode × diagram_type × summary_type × slug) combinations and fails when a combination that should have been rendered came out empty for non-analyzer reasons.StigAnalysisP3SelectProvenanceCompileCommandsTask'sfailOnEmptyFocusprecondition — the task fails before doing real work if its own focus set is empty.
Consumer-side (Case 2)
The proposed
stigAnalyzeVerifyToolchainVersions(issue 16 C.1c): comparesclang --versionand analyzer--versionagainstpins.env. Neither the bitcode producer nor the analyzer can do this check alone; the consumer that depends on both can.stigAnalysisP5ApplyBitcodeTransformsowns final-module undefined-symbol classification and writesunresolved_application_symbols.jsonplusallowed_external_symbols.jsonbeside final P5. The explicitstigCheckP5SymbolCompletenesstask turns those producer-owned artifacts into the automatic P6 gate.StigAnalysisP6RunAnalyzerTask.verifyRequiredSymbolsPresent()repeats the concrete launch precondition against the exact module/configuration it is about to execute. This is deliberate defense in depth: the P5 gate reports an application-incomplete module, while the P6 check reports that the imminent analyzer invocation is not runnable.
Anti-pattern: dual checks
A check duplicated on both sides is worse than one in the right place: when one side regresses, the other catches it but the failure message points at the wrong layer. If you find yourself adding a check on the consumer that's "really" the producer's job, move it to the producer instead. The only acceptable duplicate is a defence-in-depth check that's explicitly noted as such and has a distinct failure message for each layer.
Anti-pattern: deferred-to-consumer when the producer knows
When the producer has all the information needed but the check happens to be "easier" to write at the consumer, write it at the producer anyway. Convenience at write time is a poor trade for debuggability at break time — and the producer fires once where the consumer may fire many times across the dependency graph.