System Assurance Panel — Implementation Report v3.1

What changed in v3.1 (post-v3 caption fixes)

Field-testing v3 surfaced six interactions between the rules matrix and real STT engines that needed reconciliation. All shipped:

A2 self-rejection bug — the policy gate was rejecting every successful commit because state==='committed' was set before policy eval. Made A2 a no-op pseudo-rule (mutation prevention is enforced upstream by CaptionStabilityEngine._startNewSegment).
A4 word-wrap evaluation — STT emits raw single-line strings; A4's 42-char limit was rejecting any caption >42 chars even though the renderer wraps them across two lines. buildEvaluationInput() now simulates the renderer's greedy word-wrap so per-line lengths reflect what the operator actually sees on the wall. Single-line 44-char captions now pass; 3+ line captions correctly fail A3.
Force-commit B1+B3 starvation — batch engines (Whisper, Groq, Google STT Final) commit on first emission with stableFrameCount=0, durationMs=0. B1 (≥3) + B3 (≥800ms) both failed → permanent DELAY → no caption ever reached the wall. _commitCurrent now normalizes both to spec-canonical thresholds when reason starts with forceCommit — engine declaration of finality counts as evidence the stability requirement is satisfied.
Sentence auto-segmentation — long multi-sentence finals like "Play nice. It's working. I think it's working." would wrap to 3+ lines and fail A3. The pipeline pre-splits on ./!/? boundaries before ingest; each sentence flows independently. Natural BBC-subtitle cadence — one digestible unit at a time.
Silent-STT watchdog — Google STT's latest_short recognizer can stay "active" but stop emitting transcripts for 30+ seconds (gRPC stream alive, audio flowing, no results). 3-tier watchdog in TranscriptionService: 0–8s normal, 8–30s warning ("STT silent for Ns"), 30s+ error + auto-restart engine (throttled 1/min).
Live caption-size revert — captionEffectsEngine.ts was multiplying user-set captionFontSize by 0.8 during silence when voice-reactive style was enabled. Slider changes appeared to "drop back to original" after 2s of quiet. Removed the size shrink from the silence path; opacity fade remains. Speaking-time voice reactivity (whisper/normal/shout, energy boost) preserved.
Inline rule IDs on tiles — Matt's "less cognitive work" feedback. Policy gate captures decisive violations (the rules that drove reject/delay), telemetry formats them as "B1+B3 (delay)" or "A5 0.45 (reject)", panel renders inline next to tile label when warning/error. No hover required to see the failing condition.

Tests: 18/18 unit tests still passing for the rules matrix (added 2 new cases for the word-wrap path).

What changed in v3

v3 lands the bulk of Matt's Follow-Up Enhancements: Failure Specificity, STT Trace, and Logging Control v2. The panel evolves from "something is wrong" to "this exact condition failed, here is the evidence, and here is the next action."

§15.1 Primary Failure Line — when global is NOT_READY, a red banner appears above the tile strip stating the exact failing condition (e.g. "Renderer not receiving committed captions"), visible without hover. DEGRADED gets an amber variant.
§15.2 DiagnosticCondition — every subsystem in warning/error tracks a structured condition: { subsystem, status, conditionId, conditionText, observedValue, threshold, firstSeenAtMs, lastSeenAtMs }. Stable conditionIds (stt.stale, renderer.no_committed_caption, policy.all_blocked, etc.) for reliable diagnostics correlation.
§15.3 Diagnosis v2 — replaced {cause, action} with { primaryFailure, subsystem, conditionId, condition, evidence[], cause, action, confidence }. Evidence array cites cross-subsystem state (e.g. "audio.RMS = 0.004", "stt.lastResultAt = ...") so the operator sees the full picture.
§15.4 Per-tile reason — tiles in warning/error show the failing condition inline next to the label (e.g. POLICY · B1+B3 (delay) · 5/19 blocked) — no hover needed.
§15.8-15.10 Logging Modes — off | errors_only | warnings_and_errors | normal | debug | trace. Default warnings_and_errors per spec. Trace mode auto-expires after 5 min with countdown chip in the LogPanel header. Indicators (telemetry tiles) stay alive at every mode.
§15.11 Subsystem Log Filters — Logs tab now has filters for All / Errors / Warn / Audio / STT / Stab / Policy / Rend / Outputs / Sys / Ports / Scope / Perf. Match wide-net regex against source/service tags.
§16 Snapshot v2 — JSON snapshot now includes eventSummary (totalEvents, bySubsystem, bySeverity, transitionCounts, mostNoisySubsystem) and patterns (statusFlapping, flappingSubsystems, staleSignals, duplicateBursts, forceFinalLoops). Claude can now spot flapping behaviour without scanning raw events.

What v3 deliberately defers

§15.5 STT Behaviour Trace + §15.6 STT Debug Snapshot — separate work. New STTTraceEvent ring buffer in transcriptionService with eventTypes (interim/final/force_final/duplicate/churn/silence/timeout/restart/error), a "Copy STT Debug Snapshot" button with built-in Claude analysis prompt (§15.7), and pattern detection (duplicateBursts, latencySpikes, forceFinalLoops). Estimated ~half-day; will land alongside the per-engine config UI.
§15.12 Performance Benchmark Mode — independent project. Reports FPS / CPU% / GPU% / memory / frame drops / caption latency / events-per-sec / disk-writes-per-sec across all 5 logging modes to prove which are safe in live performance.
Per-engine config UI / control surface — Matt + Jack agreed (2026-05-04) that the panel must evolve into a control surface: per-engine STT knobs (forceAcceptMs, confidence floor override), per-profile policy thresholds (A4 chars/line, A3 lineCount), profile picker per layer, all preset-persistable. Saved as a separate plan; needs joint design pass on the knob taxonomy before code.

What changed in v2 (still live)

v1 stood up the panel UI, telemetry singleton, freshness checks, Safe Mode toggle, raw-STT leakage gate, and on-disk snapshots — all listening for signals that nothing was emitting.

v2 wires the captioning pipeline so those signals actually fire. Matt's Caption Policy Rules Matrix v2 (May 2026 spec) is implemented in full: 6 Class A rules, 4 Class B rules, 3 Class C rules, 1 Class D rule, plus the §7 decision flow (A fail → reject; B fail → delay; B4 force-commit window at >3000ms when A passes; C as warnings). The CaptionPipeline facade is now instantiated per STT session whenever any targeted text layer has captionStabilityEnabled = true, routing transcripts through CaptionStabilityEngine + CaptionPolicyGate before reaching the wall.

STAB and POLICY tiles now light up. Safe Mode auto-engages on repeated Class A failures (>50% reject ratio over rolling 20 evaluations). 16 unit tests covering spec §16 (Class A/B individual rules, valid commit, force-commit window F1, reject-after-timeout F2, correction event X1, precedence P1, determinism D1, Class C warnings) all pass.

Summary

Real-time operational cockpit for the captioning pipeline lives at the top of the Logs tab in the WallSpace desktop app. Five subsystem tiles (Audio / STT / Stability / Policy / Renderer) plus a global READY / DEGRADED / NOT READY badge surface the state of every link in the chain. Diagnostic Reasoning correlates failures into probable cause + suggested action. Safe Mode is real (not just a label) and gates the renderer to commit-only when engaged. Snapshot JSON copies to clipboard on demand and writes to disk automatically when the system enters NOT READY. Class A/B rules now drive caption commit decisions deterministically per Matt's matrix.

Net new surface in v2: captionPolicyGate.ts rewritten end-to-end (~280 LOC), captionStability.ts types extended with PolicyDecision / PolicyResult / CaptionEvaluationInput / buildEvaluationInput() helper, captionPipeline.ts updated to consume the new {decision, violations} result shape, App.tsx integration into makeTextCallback (wraps with pipeline routing when stability flag is on for any targeted layer), captionPolicyGate.test.ts (16 tests) under tests/unit/.

What it looks like

READY — all green

READY AUDIO STT STAB POLICY REND Safe · Debug · ⇪ Snapshot · 📁

DEGRADED — STT latency warning

DEGRADED AUDIO STT STAB POLICY REND

NOT READY — diagnosis surfaces

NOT READY AUDIO STT STAB POLICY REND

Cause: Audio input failure preventing STT.
Action: Check mic/device permissions before restarting STT.

Phase-by-phase status

Phase	What	Status	Files
1	Skeleton: telemetry singleton + hook + panel + Logs-tab mount	Done	`services/assuranceTelemetry.ts`, `hooks/useAssurance.ts`, `components/AssurancePanel.tsx`, `App.tsx`
2	Status wiring: 5 subsystems push real signals (4Hz audio heartbeat, STT latency, stability churn rolling window, policy block ratio, renderer frame age + raw-STT leak detector)	Done	`audioEngine.ts`, `transcriptionService.ts`, `captionStabilityEngine.ts`, `captionPolicyGate.ts`, `captionRenderer.ts`
3	Diagnostic Reasoning: 5 IF/THEN correlation rules implemented verbatim per spec; rolling 50-event log appended on every status transition	Done	`assuranceTelemetry.ts` (`evaluateDiagnosis()`)
4	Copy Diagnostic Snapshot: clipboard JSON matching spec shape (system, subsystems, captionState, STT, stability, policy, renderer, diagnosis, events)	Done	`assuranceTelemetry.getSnapshot()`, panel header button
5	UI refinement: 150ms tile color transitions, click-to-expand detail row, `aria-live="polite"` + `role="status"`, ≤2s glance-readable layout	Done	`AssurancePanel.tsx`
6	Safe Mode: real wiring (not a label). Toggle in panel + auto-engage when STT/Policy stress sustained for 10s; commit-only gate at `_emitIfAllowed()` filters provisional/stabilising captions out of the renderer.	Done	`captionStabilityEngine.setSafeMode()`, `captionPipeline.ts` bridge, telemetry auto-engage rule
7	Raw-STT leakage gate. Enforces "Renderer must NEVER display raw STT output" by short-circuiting `renderCaption()` for non-committed presentation states. Shipped behind `debugMode` toggle for one-release behavior comparison; flag will be removed next release.	Done	`captionRenderer.ts:631` (replaced opacity-fade with hard gate)
8	Auto-snapshot on degradation: writes JSON to `~/Library/Application Support/crt-wall-controller/diagnostics/` when global state enters NOT READY (rate-limited 1/min, prunes to 50 files). "Open Diagnostics Folder" button in panel header.	Done	`main/diagnosticsBridge.ts`, preload IPC, telemetry auto-trigger
9	This report — v2.	Live	`assets/web-portal/system-assurance-panel-report.html`
10	Rules Matrix v2 — full A/B/C/D rule set implemented per Matt's spec; new types (`PolicyDecision`, `PolicyResult`, `CaptionEvaluationInput`); decision flow per §7 (A → reject; B → delay; B4 force-commit at >3000ms; C as warnings).	Done	`captionPolicyGate.ts` rewritten, `types/captionStability.ts` extended
11	CaptionPipeline integration — pipelines instantiated per STT session keyed by `engine:source` when any targeted layer has `captionStabilityEnabled=true`. Lifecycle disposed when session ends. Pipeline.onCommitted fires existing delivery (transcript save, OSC, Scope, translation, etc.) only when rules pass.	Done	`App.tsx` `makeTextCallback` + `captionPipelinesRef`
12	Safe Mode auto-engage on Class A failures — >50% reject ratio over rolling 20 evaluations triggers Safe Mode (per spec §11). Existing STT-error and churn-warning triggers retained.	Done	`assuranceTelemetry.ts` `classARejectRatio`
13	Unit tests (spec §16) — 16 tests covering A1-A6, B1-B3, C1, F1 (force-commit), F2 (reject-after-timeout), X1 (correction), P1 (precedence), D1 (determinism). All passing.	Done	`tests/unit/captionPolicyGate.test.ts`

Rules Matrix v2 — what's live

Rule	Class	Condition	Action
A1 No Raw STT	A	caption not committed	MUST NOT render (renderer gate, Phase 7)
A2 No Mutation After Commit	A	caption is committed	REJECT
A3 Maximum Lines	A	lineCount > 2	REJECT
A4 Maximum Chars/Line	A	any chars/line > 42	REJECT
A5 Minimum Confidence	A	confidence < 0.60	REJECT
A6 Meaning Stability	A	meaningChangeRisk > 0.30	REJECT
B1 Stability Requirement	B	stableFrameCount < 3	DELAY
B2 Churn Limit	B	churnCount > 3	DELAY
B3 Min Display Duration	B	durationMs < 800	DELAY
B4 Max Stabilisation Window	B	not committed AND durationMs > 3000	FORCE commit if A passes, else REJECT
C1 Preferred Line Length	C	chars/line > 36	ALLOW + warning
C2 Punctuation Completeness	C	missing trailing punctuation	ALLOW + warning
C3 Phrase Completion	C	sentence ends mid-phrase	ALLOW + warning
D1 Capitalisation	D	first char not capitalised	ALLOW (no decision impact)

Spec compliance checklist

5 subsystem tiles in fixed order: AUDIO → STT → STABILITY → POLICY → RENDERER
Global state derivation: any error → NOT READY; any warning → DEGRADED; else READY
Per-subsystem freshness thresholds: audio 500ms/2s, STT 2s/10s, stability 1s/5s, policy 2s/10s, renderer 500ms/3s
200ms debounce on status transitions to prevent flicker
≤200ms refresh rate (250ms internal tick)
≤2-second glanceability (tile strip always visible, no interaction required)
Diagnostic Reasoning: all 5 IF/THEN rules from §6 implemented
"Copy Diagnostic Snapshot" button → JSON to clipboard
Rolling event log: last 50 transitions, newest first, severity-coded
Safe Mode active = renderer receives committed captions only
Raw STT output never reaches renderer in normal mode (Phase 7)
No hidden state — every internal subsystem status is externally visible
No silent failures — auto-snapshot writes JSON on every NOT READY transition
aria-live="polite" + role="status" for screen-reader announcements

Diagnostic Reasoning rules (verbatim)

1. audio.error AND stt.error
   → Cause:  Audio input failure preventing STT.
   → Action: Check mic/device permissions before restarting STT.

2. audio.ok AND stt.error
   → Cause:  STT engine, API, or connection failure.
   → Action: Restart STT engine; check API key and network.

3. stt.ok AND renderer.error
   → Cause:  Caption pipeline or renderer failure.
   → Action: Inspect CanonicalCaptionLite creation and renderer input.

4. stability.warning AND detail contains 'churn'
   → Cause:  Unstable STT hypotheses.
   → Action: Increase stability threshold, reduce update rate, or use final-only mode.

5. policy.warning OR policy.error
   → Cause:  Captions rejected by confidence/churn/meaning rules.
   → Action: Inspect policy violations and active preset thresholds.

Snapshot JSON shape

{
  "ts": 1714780800000,
  "system": { "global": "DEGRADED", "safeMode": false, "debugMode": false },
  "subsystems": {
    "audio":     { "status": "ok",      "lastUpdate": ..., "detail": "RMS 0.042" },
    "stt":       { "status": "warning", "lastUpdate": ..., "latencyMs": 1820,    "detail": "Latency 1820ms" },
    "stability": { "status": "ok",      "lastUpdate": ..., "detail": "Stable 7, sim 0.94" },
    "policy":    { "status": "ok",      "lastUpdate": ..., "detail": "1/20 blocked" },
    "renderer":  { "status": "ok",      "lastUpdate": ..., "detail": "wall-banner" }
  },
  "captionState": "stabilising",
  "stt": { "engine": "web-speech", "lastResultAt": ..., "latencyMs": 1820 },
  "diagnosis": null,
  "events": [
    { "ts": ..., "subsystem": "stt", "from": "ok", "to": "warning", "message": "Latency 1820ms" },
    ...
  ]
}

Known gaps / not done

Awaiting your inputs, Matt

Profile switching UX — Compliance / Accessible Enhanced / Live / Immersive / Experimental. Not in this round; needs joint design. Panel currently treats whatever profile the stability engine has loaded as canonical (per-engine defaults: Google, Whisper, Groq, Moonshine, Sherpa).
Per-layer profile selection UI — currently captionStabilityEnabled is a binary checkbox per text layer. A profile picker (Compliance vs Live vs Immersive) would map onto your 5-profile architecture from vocal-forensics-v3.
Token-level policy — A6 currently uses the segment-level meaningChangeRisk. Token-level critical-term locking (numbers, negation, named entities) per your Caption Accessibility Framework would tighten A6 further.

Future work

Live remote streaming of Assurance state to wallspace.studio — currently the only remote handoff is the JSON snapshot file. A real-time dashboard at, e.g., /assurance would let you watch a venue's pipeline state from elsewhere. Separate networked-diagnostics project.
Long-term time-series telemetry — only NOT READY transitions persisted on disk today. A continuous metric trail (audio levels, STT latency over time, stability churn distribution) would feed the kind of post-event analysis the Caption Stability Report architects.
Decision Log cross-system bridge to A.EYE.ECHO — Phase 4 of the vocal-forensics-v3 plan. Echo and WallSpace currently maintain independent decision logs; unifying them is the path to "Decision Log as cross-system common language" from your spec.
Phase 7 flag removal — raw-STT-gate ships behind a debugMode toggle for one release so you (and future operators) can A/B compare provisional-fade vs. final-only on the wall. Next release removes the toggle and final-only becomes the only behavior.

Spec items deliberately deferred

Replacing the LogPanel "Errors" filter tab — your spec says the panel must be self-sufficient (no log-dependency for primary interpretation), and it is. Logs stay as deeper diagnostic reference.
5-state caption-pipeline visualization (provisional/stabilising/committed/degraded/safe_mode) as its own widget — currently surfaced as a single field in the Layer 3 detail. If you want a proper state-machine viz, it's a small follow-up.

Implementation file inventory

File	LOC	Role
`src/renderer/services/assuranceTelemetry.ts`	~300	Singleton store, debounce, freshness, rules engine, snapshot, auto-engage Safe Mode
`src/renderer/hooks/useAssurance.ts`	8	React subscription hook
`src/renderer/components/AssurancePanel.tsx`	~180	Layer 1+2+3+4 UI
`src/main/diagnosticsBridge.ts`	50	IPC for snapshot file write + open folder
`src/renderer/services/audioEngine.ts`	+40	4Hz heartbeat → reportAudio
`src/renderer/services/transcriptionService.ts`	+20	reportSTT on status + result
`src/renderer/services/captionStabilityEngine.ts`	+25	reportStability + setSafeMode
`src/renderer/services/captionPolicyGate.ts`	+15	reportPolicy with rolling block ratio
`src/renderer/utils/captionRenderer.ts`	+30	reportRenderer + Phase 7 commit-only gate
`src/renderer/services/captionPipeline.ts`	+15	Bridge Safe Mode telemetry → engine
`src/main/index.ts` + `preload.ts`	+10	Bridge registration + IPC surface
`src/renderer/App.tsx`	+5	Mount AssurancePanel above LogPanel in Logs tab

Next steps for review

Enable captionStabilityEnabled on a text layer, drive it with Google STT, and watch STAB + POLICY tiles light up. Verify duplicate captions (the prior Force-final repeat-emit pattern) are now suppressed by A2.
Validate the rules matrix against your test corpus — run npx vitest run tests/unit/captionPolicyGate.test.ts for §16 cases. All 16 currently pass; if you add more spec cases I'll wire them too.
Confirm the 0.60 confidence floor (A5) is right for your engine mix — Google STT typically reports 0.5–0.95, Whisper doesn't report at all (defaults to 1.0 trust). May need per-engine tuning.
Decide on profile-switching UX (5 profiles from vocal-forensics-v3) — needs joint design before code.
Decide whether to keep the Phase 7 debugMode flag for >1 release or drop it sooner — current plan removes it next release.
Tell us where the next slice should land: (a) profile-switching UI, (b) token-level critical-term locking, (c) live remote dashboard at wallspace.studio/assurance, (d) Decision Log bridge to Echo.