Source-code analysis pipeline

What P3 includes: a source-file example

P3 compiles two sets of original source files, using their original headers:

1. softwareUnderAnalysis native compilation units: the source files explicitly selected to contribute original implementation code to P5, such as activity.cpp.

2. Transform-owner provenance compilation units: original source files containing the function or global definitions selected by active activity transform rules. For example, logger.cpp supplies the original log() implementation when a rule will replace log() with a model in P5. The rule's originalCompileUnit names that file.

In this example, P3 contains compiled original code from both activity.cpp and logger.cpp. We retain the original log() body so that we can compare its behavior with the configured P5 model. If logger.cpp was included only for this comparison, that does not select its original implementation for P5. P5 compiles the selected activity code with its analysis harness, stubs, and model sources, then applies the configured LLVM rewrites.

Each compilation unit is a whole source file with its included headers, so compiling logger.cpp can also include other function definitions from that file. P3 does not recursively compile every other file called by those functions. Where a required original implementation is absent from P3, inspect the initial LLVM module before deciding that the original definition is missing. A declaration supplies a function's name and type, but no body to review.

For initialized analysis, provenanceBuild { ... } is normally empty: STIG derives these two sets from softwareUnderAnalysis and the active rules' originalCompileUnit values. P3 retains the original definitions; the analysis harness, analysis stubs, and configured replacements enter through P5.

What makes a rerun reusable

With nativeBuild { bazel { trackActionInputs = true } }, P1 asks Bazel to build and inspect the target on every invocation. Bazel reuses unchanged native work. STIG records a deterministic fingerprint of the target/options, compile actions, command arguments, action environment, output paths, and input contents, including headers and generated inputs, in build/stig/stigAnalysisP1BuildNativeTarget/native_action_inputs_fingerprint.json. The file changes only when that identity changes. Downstream tasks also track their own configuration, toolchain, harness/model sources, and other declared inputs.

This can avoid repeating expensive LLVM compilation and analysis for each activity. Identical native binary bytes alone are insufficient: comments, LLVM options, selected units, harness code, rewrite rules, or analyzer settings can change without changing the native executable. Reuse depends on the relevant task's complete input identity and available outputs/cache entries; the time saved depends on the activity. Setting trackActionInputs = false skips the native build and disables this native-input tracking.

Concurrent activity compilation

Each selected activity retains its own pipeline. Bazel commands share an exclusive Gradle service and Bazel's native build cache; each Bazel build can execute its compilation actions in parallel. Unselected activities do not become dependencies of the selected activity.

Before releasing Bazel access, stigAnalysisP1ExtractNativeCompileCommands copies the native compile actions' recorded inputs and the configured targets' declared C++ headers into its activity's compilation_inputs/ output directory. Both the native-input fingerprint and the snapshot use cquery to read CcInfo header declarations across the selected targets' dependency closure. This includes headers omitted from aquery by Bazel input discovery, including with Bazel 7. The capture includes generated sources and headers and the contents reached through Bazel symlinks. manifest.json records original paths and content hashes. These copies are Gradle outputs, and LLVM build tasks track their bytes as inputs. A missing declared input fails P1.

P2 supplies Clang commands using the captured sources, headers and working directory. Clang path mappings preserve original source names in debug information and macros. Its virtual filesystem redirects absolute native includes to captured files. LLVM builders verify compiler dependency files and reject any access to uncaptured files in the original native workspace or Bazel directories. Consumer harness/model sources remain separately tracked Gradle inputs; host tools retain the existing pinned toolchain policy.

P7 metrics and function descriptions resolve captured files through the snapshot's recorded VFS aliases. They read captured contents for comments and line numbers, while source selection, report identities and code links refer to the original files. Header aliases resolve to the same original source identity. Function description collection scans captured native headers when a snapshot is present. For a repository imported through git subrepo, source-browser paths start at its .gitrepo directory and use the recorded imported commit. CommitRef.PARENT also keeps that imported commit; the enclosing checkout's revision identifies a different repository. IntelliJ IDEA links use the enclosing checkout's actual origin and include the import directory in the local path.

P3, P5 and both LLVM inventory builders can therefore compile different activities concurrently, including while Bazel prepares another activity. Original-source inspection tasks retain exclusive access. P6 reads finalized LLVM and is scheduled independently through maxParallelAnalyzerExecutions.

The usual root stigGenerateHtml and selected activity task paths work unchanged. No cross-activity ordering rules or additional selection switch are required.

Independent Gradle invocations must not replace the same checkout's snapshots or report outputs concurrently. stigLockAnalysisWorkspace acquires an operating-system lock in .gradle/stig-analysis-workspace.lock before analysis, report generation or clean tasks can change those files, including restoration from the Gradle cache. A second invocation fails with an explanation before entering those tasks. Wait for the first invocation, or select the required activities together in one invocation. The lock stays held through downstream report tasks and is released when the build finishes; its empty file remains. This guard does not limit activity parallelism within one invocation and does not coordinate separate consumer checkouts pointing at the same native workspace.

What the original-code reference establishes

P2 converts compiler commands; it does not compile bitcode or translate a GCC executable into LLVM. The full LLVM, P3, and P5 input builders compile their own selected sources. P3 is not cut out of the linked full module, and P5 is not simply a rewritten copy of P3: its implementation selection and support sources differ.

The pinned Clang compilation resolves includes, macros, conditional compilation, overloads, and templates for those commands. LLVM text exposes the resulting operations, while debug information links them to source locations and types. The common recipe preserves that information and uses optimization/inlining and disabled vectorizers to produce operations the analyzer can interpret.

Use the initial LLVM module as the original-code reference for the captured build configuration. It does not by itself prove equivalence with GCC machine code or cover every runtime dependency: unavailable compilation actions, prebuilt libraries, and shared-library implementations can leave definitions absent. Its scope is captured product compilation units, not every repository file or the native linker's exact surviving-function set. P3 is a selected reference; absence there cannot establish absence in the full original program.

Where metrics and source comments come from

P7 reads selected source files using compilation commands. sourceCodeSelection.from, complexityMetrics.from, and functionDescriptions.from choose those command providers. With sourceCodeSelection { from(analyzerInput.compileCommands) }, the P7 branch starts at the P5 command-selection task. stigAnalysisP7CollectMetrics runs source-level metrics through Clang tooling; stigAnalysisP7CollectDescriptions extracts function documentation from source comments. Neither extracts comments from P3 bitcode or needs P3 compilation just for this purpose. P5's separate ir_function_metrics.json describes LLVM functions and is distinct from these source metrics.

The annotated task flow shows these branches, with one task per box and artifacts on the arrows. In the current initialized workflow, P6's transform-review input also brings the full and configured-input inventories into its prerequisites; they can be built or reused during a report run, as well as during explicit configuration work.

P4 — software under analysis

stigAnalysisP4SelectSoftwareUnderAnalysis emits the configured softwareUnderAnalysis native compilation-unit list for P5. Transform rules can separately supply additional implementations. P4 does not compile or link anything. The DSL owner is softwareUnderAnalysis { ... }:

includeTargetSourceTree intersects logical repository paths with the configured native target's compile actions. It does not scan arbitrary files or include another Bazel target solely because a path prefix matches.

P5 — analysis harness, stubs, and native model sources

The DSL owner is analyzerInput { ... }:

For a generated lifecycle harness, P5 validates the compiled harness contract: the entry remains a noinline definition retained by llvm.compiler.used, lifecycle calls remain direct and ordered, and generated source/sink marker definitions and calls agree with the contract.

P5 symbol-completeness gate

The per-activity stigCheckP5SymbolCompleteness task classifies unresolved symbols in the final transformed module. Configured runtime/standard-library externals are allowed; missing referenced application implementations are HIGH findings. P6 depends on this check automatically. With failAt=CRITICAL, those findings permit explicitly provisional execution; invalid LLVM, tool failures, and missing required run-plan symbols remain execution blockers. The root task is an explicit aggregate over all source-analyzed activities.

This gate reports required application-definition gaps. It does not establish transform semantics or original-to-P5 agreement; review those through Check and align analysis evidence. An allowed-external category does not supply a function body or an analyzer model. Calls remain in LLVM; the selected analysis applies its available summaries and external-call handling. For example, permitting C/GNU floating-point environment functions does not model CPU exception flags, writes to saved flag objects, or exception-derived flows. When those effects matter to the requested result, they need explicit modeling or must remain a stated analysis limitation. Classifying an external dependency does not classify its application callers as runtime functions. See External symbol categories for the Kotlin classification source, category descriptions, configured-ignore defaults, and unused declarations.

P6 — analyzer execution

The DSL owner is analyzerExecution { ... }:

maxParallelAnalyzerExecutions limits native analyzer processes across the build. solverWorkers caps independent source solves inside each process; those workers share the build's logical-CPU-count-minus-one budget. Each activity executes its run-plan entries sequentially. These are separate limits: two processes with eight source workers each consume two process slots and sixteen CPU slots.

On Linux, maxAllowedRamAndSwapMiB is the memory allowance per solver worker. Before launch, the coordinator grants the minimum of the requested workers, runnable source/sink jobs, free CPU slots and available RAM divided by this allowance (rounded down). It reserves memory and CPU slots together across all activities. The process's systemd MemoryMax ceiling equals its granted workers multiplied by the allowance. If even one worker cannot fit, the run waits without holding CPU slots. It reserves 5% of physical RAM as host headroom and excludes swap from admission capacity. Reservations account for future growth even before a process allocates its memory. Resident anonymous memory already consumed by a tracked analyzer is credited against that reservation, since Linux already subtracts it from available RAM. File cache and swap are not credited. Reservations are released after process/scope cleanup, including failure and cancellation; available RAM is rechecked while waiting for other host work to finish.

For example, with 70 GiB available after headroom and a 20-GiB allowance per worker, at most three workers can be admitted across all activities. One process can receive three workers and a 60-GiB ceiling, or one can receive two workers with a 40-GiB ceiling while another receives one with a 20-GiB ceiling. A fourth worker waits. At 40 GiB per worker, a process granted three workers reserves and enforces 120 GiB. Their combined workers still obey the shared CPU budget. The worker grant stays fixed until process exit. An allowance exceeding physical RAM minus headroom fails immediately. A missing or invalid Linux memory inventory also fails admission. resource-admission-wait logs the reason for waiting, and each executed run records its reservation, allocated workers and admission wait separately from analyzer execution time. memoryPerSolverWorkerMiB and processMemoryMaxMiB distinguish the configured allowance from the resulting process ceiling in run logs.

This changes the setting's earlier per-process meaning: existing configured values now apply per worker.

null explicitly disables the memory reservation and ceiling. On other operating systems only the existing CPU coordination applies. Reservations are shared within one Gradle invocation, not across independent builds. The memory ceiling covers all workers and shared analysis state in one process; include process overhead in the per-worker allowance. Threads do not have individual enforced limits. Exceeding the process ceiling fails that run rather than reducing its analysis scope or retrying with more memory. systemd still accounts swap separately from MemoryMax; this setting does not configure MemorySwapMax.

maxSinksPerSolver controls deterministic sink batching inside one process; null keeps all configured sinks in each source-isolated solve. Sink verdicts are ORed across covering solves, while positive reconstructed DAG components retain their solve/source/sink provenance and are never unioned into a synthetic authoritative tree.

P6 checks that its configured transform fingerprint agrees with final P5 rewrite/body-validation evidence. P6-stage inconsistency is reported by P6; it is not imported opportunistically into a different check based on a stale file in build/.

P7 — source descriptions, metrics, and diagrams

stigAnalysisP7GenerateDiagrams generates the Sigma, SVG, and Cytoscape diagrams directly from analyzer results. stigGenerateDiagramAppendix assembles the analysis appendix and publication assets.

sourceCodeSelection selects the source files from which P7 reads descriptions and computes metrics. It does not change the code that P6 analyzes:

Run the analyzer from transformed inputs

Use the existing activity P6 task to measure an analyzer against retained inputs:

--from also accepts an extracted evidence directory containing evidence_manifest.json; ZIP creation is optional. The activity task selects its own exact Gradle path from a multi-module collection. Relative option paths resolve from the consumer root. Omit --run to execute all captured runs, or select one exact run name such as --run=cascading-faults-B. See the generated task reference for the option schema.

This mode removes the local P1–P5 producers, P6 slicing and transform-review producers from the task graph. Supplying --analyzer-executable also skips analyzer build/install; the executable's reported identity and SHA-256 are recorded. Without that option, the normal local-sibling/pinned analyzer resolution applies. Gradle still loads the plugin, so a changed included STIG build may compile plugin code before selecting P6.

STIG validates manifest paths, sizes and hashes before using the input. A configuration checkpoint needs final llvm/p5/module.bc and module.ll, run-plan/stig_analyzer_runs.json, and the matching configurations and .endpoints.json sidecars under run-plan/all/. New stigCheckAnalysis --configuration-only collections capture these without running P6. Older configuration ZIPs that contain only LLVM text cannot establish this executable checkpoint and are rejected with the missing input named. An arbitrary build-output directory or a loose bitcode file is insufficient.

A captured sliced run plan requires selected_p6_modules.json and its exact .bc/.ll pairs, including any P6 slices. It never substitutes the unsliced P5 module. When previous analyzer JSON is present, its recorded settings are also checked. Source/sink selections, return/parameter annotations, solver, alias analysis, call-graph mode, path-feasibility mode and sink batching remain captured inputs. Repository mappings remain as captured.

Timeout, memory limit, execution concurrency and diagnostic sampling use the invoking project's analyzerExecution settings. They are recorded separately from the captured analysis configuration. Output contains transformed_input.json, stig_analyzer_run_log.json, the required copied input files, and an analyzer/<run>/ directory with JSON, logs and sink_summary.json. Each configured sink is listed, including clean sinks. Failed runs retain their logs; the task stops at the first failed run. Measurements execute afresh even with Gradle's build cache enabled; use distinct output directories for baseline and candidate runs. A populated destination must already belong to this task.

These are diagnostic results kept outside normal P6/report outputs. Finishing successfully establishes execution and output validity; findings such as cascading faults or missing expected flows still need review. A diagnostic result does not become a current P7/HTML report. Gradle rejects a combined transformed-P6 and P7/report invocation for the same activity, so existing local results cannot silently stand in for the diagnostic run. Omit all transformed-input options for the ordinary local pipeline. full and reduced starting checkpoints remain unsupported.

Use Check and align analysis evidence for collection options, review steps, and the limits of archived replay.