feat(fixtures): curated demo targets + seed script + golden PLC baselines (#187) #232

Merged
sharang merged 1 commits from feat/demo-target-fixtures into main 2026-08-31 15:58:17 +00:00
Owner

Closes the fixture half of #187 (story of epic #186). Sets up the baseline #188 (nightly regression) will consume.

What

  • fixtures/demo-targets/targets.json — 5 curated targets covering the story's scope:

    • plc-pump-station — PlcSps composite: pump_station.st + pump_fbd.xml + optional Modbus live URL (DEMO_PLC_MODBUS_URL, in-cluster plc-sim) + optional firmware image (DEMO_PLC_FIRMWARE_IMAGE)
    • plc-conveyor-line — PlcSps pure SAST: conveyor.xml + traffic_light.st
    • git-cra-vuln-demo — plain git SAST (sharang/cra-vuln-demo, pinned 2195124)
    • web-juice-shop — WebApp: juice-shop/juice-shop @ v19.2.1 + live URL (DEMO_WEB_URL, default http://localhost:3000)
    • firmware-zephyr-example — FirmwareRtos: zephyrproject-rtos/example-application (pinned 38a6d9b)

    Every target has an expect golden baseline. PLC baselines were recorded from the real analyzer (16 findings / 8 rules; 8 findings / 5 rules).

  • compliance-agent::fixtures — typed loader + two --lib tests: manifest well-formedness (unique keys, uploads exist, PLC has plc_format, git has branch+pin, all four target families present) and the offline PLC golden-baseline check via analyze_tree. Runs in the existing CI test job.

  • scripts/seed-demo-targets.sh — curl+jq seeder over the public API (POST /targets/artifacts/upload/detect → optional /scan), --only, --reset (deletes exactly the Demo · prefix), env overrides for infra-dependent artifacts so it seeds cleanly on a laptop.

  • docs/guide/demo-targets.md + sidebar entry: what's in the set, reproducibility rules, how to (re)onboard manually.

Acceptance criteria (#187)

  • Fixtures versioned and reproducible (checked-in files + pinned commits + seed script)
  • Documented how to (re)onboard each as a target
  • Consumed by the nightly regression — that is #188 (next PR)

Verified

  • cargo fmt --all --check ✓ · cargo clippy -p compliance-agent -- -D warnings ✓ · cargo test -p compliance-agent --lib fixtures:: 2/2 ✓
  • bash -n scripts/seed-demo-targets.sh ✓ (not run against a live agent from this box)

Notes

  • The agent clones branch; pin records the commit the baseline was taken against and is enforced by convention (re-pin when updating expect), not by the clone.
  • No public firmware image is checked in; the composite target takes one via env. Filling that gap is tied to #151.

🤖 Generated with Claude Code

https://claude.ai/code/session_01EgxGHn22YEfQz5fLHSHkLv

Closes the fixture half of #187 (story of epic #186). Sets up the baseline #188 (nightly regression) will consume. ## What - **`fixtures/demo-targets/targets.json`** — 5 curated targets covering the story's scope: - `plc-pump-station` — PlcSps **composite**: `pump_station.st` + `pump_fbd.xml` + optional Modbus live URL (`DEMO_PLC_MODBUS_URL`, in-cluster `plc-sim`) + optional firmware image (`DEMO_PLC_FIRMWARE_IMAGE`) - `plc-conveyor-line` — PlcSps pure SAST: `conveyor.xml` + `traffic_light.st` - `git-cra-vuln-demo` — plain git SAST (`sharang/cra-vuln-demo`, pinned `2195124`) - `web-juice-shop` — WebApp: `juice-shop/juice-shop` @ v19.2.1 + live URL (`DEMO_WEB_URL`, default `http://localhost:3000`) - `firmware-zephyr-example` — FirmwareRtos: `zephyrproject-rtos/example-application` (pinned `38a6d9b`) Every target has an `expect` golden baseline. PLC baselines were **recorded from the real analyzer** (16 findings / 8 rules; 8 findings / 5 rules). - **`compliance-agent::fixtures`** — typed loader + two `--lib` tests: manifest well-formedness (unique keys, uploads exist, PLC has `plc_format`, git has `branch`+`pin`, all four target families present) and the offline PLC golden-baseline check via `analyze_tree`. Runs in the existing CI test job. - **`scripts/seed-demo-targets.sh`** — curl+jq seeder over the public API (`POST /targets` → `/artifacts/upload` → `/detect` → optional `/scan`), `--only`, `--reset` (deletes exactly the `Demo · ` prefix), env overrides for infra-dependent artifacts so it seeds cleanly on a laptop. - **`docs/guide/demo-targets.md`** + sidebar entry: what's in the set, reproducibility rules, how to (re)onboard manually. ## Acceptance criteria (#187) - [x] Fixtures versioned and reproducible (checked-in files + pinned commits + seed script) - [x] Documented how to (re)onboard each as a target - [ ] Consumed by the nightly regression — that is #188 (next PR) ## Verified - `cargo fmt --all --check` ✓ · `cargo clippy -p compliance-agent -- -D warnings` ✓ · `cargo test -p compliance-agent --lib fixtures::` 2/2 ✓ - `bash -n scripts/seed-demo-targets.sh` ✓ (not run against a live agent from this box) ## Notes - The agent clones `branch`; `pin` records the commit the baseline was taken against and is enforced by convention (re-pin when updating `expect`), not by the clone. - No public firmware *image* is checked in; the composite target takes one via env. Filling that gap is tied to #151. 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_01EgxGHn22YEfQz5fLHSHkLv
sharang added 1 commit 2026-08-31 12:33:28 +00:00
feat(fixtures): curated demo targets + seed script + golden PLC baselines (#187)
CI / Check (push) Skipped
CI / Check (pull_request) Failing after 3m1s
CI / Detect Changes (pull_request) Skipped
CI / Deploy Agent (pull_request) Skipped
CI / Deploy Dashboard (pull_request) Skipped
CI / Deploy Docs (pull_request) Skipped
CI / Deploy MCP (pull_request) Skipped
3ac4b34c29
Versioned, reproducible set of representative targets so every scan path can
be exercised repeatably and the nightly regression (#188) has a baseline.

- fixtures/demo-targets/targets.json: 5 targets — PlcSps composite
  (pump_station.st + pump_fbd.xml + optional Modbus live URL + optional
  firmware image), PlcSps pure (conveyor.xml + traffic_light.st), plain git
  SAST (sharang/cra-vuln-demo, pinned), WebApp (juice-shop v19.2.1 + live
  URL), FirmwareRtos (zephyr example-application, pinned). Each carries an
  `expect` golden baseline (min_findings, sast_rule_ids, cwes, control_refs,
  min_sbom_components, scans_offered, pentest_supported, detected_facts).
- compliance-agent::fixtures: typed loader (DemoTargets/DemoTarget/
  DemoArtifact/Expect) + lib tests that keep the manifest well-formed and
  assert the PLC baselines offline by running analyze_tree over the checked-in
  fixtures (runs in the normal --lib CI job).
- scripts/seed-demo-targets.sh: curl+jq seeder over the public onboarding API
  (create → upload → detect → optional --scan), --only, --reset (deletes only
  the "Demo · " prefix), env overrides for infra-dependent artifacts.
- docs/guide/demo-targets.md + sidebar entry.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EgxGHn22YEfQz5fLHSHkLv
sharang merged commit 8dcdad9fe7 into main 2026-08-31 15:58:17 +00:00
Sign in to join this conversation.
No Reviewers
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: sharang/compliance-scanner-agent#232