Skip to content

fix: preserve precheck causes and classify unreadable ref audits (#510) - #513

Merged
datacas merged 1 commit into
mainfrom
fix/510-precheck-real-cause
Nov 18, 2026
Merged

datacas merged 1 commit into
mainfrom
fix/510-precheck-real-cause

Conversation

@datacas

@datacas datacas commented Nov 18, 2026

Copy link
Copy Markdown
Owner

Closes #510.

Diagnosis

Pre-dispatch ref-audit exceptions were converted to generic strings. The dispatch
layer then treated an unreadable publication guarantee as a read-only contract
failure. The lower-level publication auditor also discarded Git stderr, so even
retaining its exception alone could not recover the actual cause. This matches
the issue's hypothesis and is a shared diagnostic failure.

Before implementation, python3 -m unittest discover -s tests -p test_precheck_causes.py exposed the missing Git stderr, missing configuration
exception message, and a bare string returned on capture failure. The module now
exercises a real Git subprocess against a missing local remote as well as injected
failures and successful retries.
The invariant is that an unreadable ref audit supplies redacted evidence, retries
once, and stops before executor invocation with publication_unverified.

Related-work search covered the issue's references, open and closed precheck/ref
audit issues, and merged precheck PRs. #508 and #511 already cover socket-blocked
isolation probes. #458's dispatch-after-consent behavior remains separate. #450
and #274 supply the surrounding capability-parity context. This implements the
root diagnostic fix within #510's scope without closing other work.

Changes

  • Preserve exception class, message and available subprocess diagnostics through
    cycle_journal.redact, including the publication auditor's Git/GitHub failures.
  • Retry read-only ref discovery/capture once; return publication_unverified
    with start_state=not_started on persistent failure. Retain failed-attempt
    evidence after a successful retry and stage artifacts after a failed post-audit.
  • Include real causes in Git-configuration, sandbox-probe, recovery and worktree
    precheck diagnostics.
  • Inventory every broad Exception/OSError handler in the five dispatch files,
    classified by whether it can swallow, must report, or leaves verification
    incomplete. A regression checks inventory coverage and handler counts.
  • Register the new module in the Windows shard manifest and its documentation
    dependency in the fast-path map. Update existing stop-result assertions to
    require the new classification and cause without starting an executor.

The stop behavior was already owned by docs/role-workspace-policy.md; this PR
documents the retry and evidence fields there and adds the requested inventory
to docs/capability-parity-audit.md. It adds no permission veto or dependency.

Work units

Unit Behaviour Files Tests Verification Rollback
WU-1 Preserve redacted causes and stop unreadable read-only audits before dispatch scripts/executors.py, scripts/publication_audit.py, scripts/run_cycle.py; both owning docs; tests/test_precheck_causes.py, tests/test_executors.py, tests/test_dispatch_failures.py, tests/test_run_cycle.py; scripts/fast-path-tests.json, scripts/windows-test-shards.json precheck, dispatch, fast-path and publication audit suites; full suite Local verification reported below independent: revert this commit

Security review

cc-security-review ran on the accumulated change. No confirmed vulnerability
was found. Reviewed surfaces: untrusted Git/GitHub stderr entering result artifacts
and reports, redaction before bounding, retry control flow, and dispatch prevention
when audit coverage is missing. No new shell interpolation or permissions are
introduced. Residual risk: journal redaction recognizes known credential formats
and is not proof that arbitrary provider diagnostics contain no sensitive data.
Live provider outages were injected; the Git nonzero-exit boundary ran locally.

Verification

Agent-reported results:

Command Exit Tests run
python3 -m unittest discover -s tests 0 2663 (1 skipped)
python3 scripts/validate-package.py 0 —
python3 -m unittest 0 0 discovered
basedpyright 0 0 errors, warnings or notes
python3 -m unittest discover -s tests -p test_precheck_causes.py 0 11
python3 -m unittest discover -s tests -p test_fast_path.py 0 42
python3 -m unittest discover -s tests -p test_dispatch_failures.py 0 30
python3 -m unittest discover -s tests -p test_consent_variants.py -k test_rev016_bash_baseline_and_real_hook_allow_and_deny 0 1

The final full run completed in 476.983 seconds. Earlier full runs exposed an
intermediate Git-helper regression, missing test registrations and old result
assertions; those were corrected. Two consent-pattern time-budget errors passed
on isolated rerun and in the final full suite. The existing POSIX-permissions
test skipped in this environment. Full schema-2 check evidence is repeated in
the stage summary comment, including the earlier failed attempts.

The full suite runs with the project's existing test isolation;
the new boundary test uses a temporary local Git repository and needs no global
Git identity, credentials or network. CI states are reported separately for the
uploaded head under the parallel CI policy.

@datacas

datacas commented Nov 18, 2026

Copy link
Copy Markdown
Owner Author

Implemented #510 at aef0cf4a6077c7748819bbf8a14b6e6363f9825d. The publication helper verified the uploaded branch, and gh pr view confirmed this exact PR head.

cc-security-review ran; no confirmed vulnerabilities were found. Redacted diagnostic flow, pre-dispatch refusal on incomplete audit coverage, and the test routing changes were reviewed.

Verification below is agent-reported. The final full suite passed (2,663 tests, one skipped); the earlier failed attempts remain in the check history. python3 -m unittest discovered zero tests. The boundary check includes a real Git subprocess failure against a disposable local remote.

{
  "tests": {
    "version": 2,
    "ran": true,
    "passed": true,
    "conclusion": "verified",
    "boundary": "required",
    "checks": [
      {
        "id": "configured-test",
        "command": "python3 -m unittest discover -s tests",
        "executor": "codex",
        "profile": "deep_coder",
        "context": "project",
        "required": true,
        "criteria": [
          "AC-1",
          "AC-2",
          "AC-3",
          "AC-4"
        ],
        "level": "test",
        "provenance": "project",
        "reason": "configured test command",
        "head_sha": "aef0cf4a6077c7748819bbf8a14b6e6363f9825d",
        "result": "pass",
        "evidence": [
          {
            "exit_code": 0,
            "executed": 2663
          }
        ],
        "cause": "",
        "attempts": [
          {
            "result": "fail",
            "evidence": [
              {
                "exit_code": 1,
                "executed": 2658
              }
            ],
            "cause": "Intermediate Git helper behavior; corrected before the subsequent runs."
          },
          {
            "result": "fail",
            "evidence": [
              {
                "exit_code": 1,
                "executed": 2663
              }
            ],
            "cause": "Missing test registrations and four old stop expectations, now fixed; two consent time-budget errors also passed on isolated rerun."
          }
        ]
      },
      {
        "id": "validate-package",
        "command": "python3 scripts/validate-package.py",
        "executor": "codex",
        "profile": "deep_coder",
        "context": "project",
        "required": true,
        "criteria": [],
        "level": "static",
        "provenance": "project",
        "reason": "configured verification command",
        "head_sha": "aef0cf4a6077c7748819bbf8a14b6e6363f9825d",
        "result": "pass",
        "evidence": [
          {
            "exit_code": 0,
            "executed": null
          }
        ],
        "cause": "",
        "attempts": []
      },
      {
        "id": "unittest-default",
        "command": "python3 -m unittest",
        "executor": "codex",
        "profile": "deep_coder",
        "context": "project",
        "required": true,
        "criteria": [],
        "level": "test",
        "provenance": "project",
        "reason": "configured verification command",
        "head_sha": "aef0cf4a6077c7748819bbf8a14b6e6363f9825d",
        "result": "pass",
        "evidence": [
          {
            "exit_code": 0,
            "executed": 0
          }
        ],
        "cause": "",
        "attempts": []
      },
      {
        "id": "basedpyright",
        "command": "basedpyright",
        "executor": "codex",
        "profile": "deep_coder",
        "context": "project",
        "required": true,
        "criteria": [],
        "level": "static",
        "provenance": "project",
        "reason": "configured verification command",
        "head_sha": "aef0cf4a6077c7748819bbf8a14b6e6363f9825d",
        "result": "pass",
        "evidence": [
          {
            "exit_code": 0,
            "executed": null
          }
        ],
        "cause": "",
        "attempts": []
      },
      {
        "id": "precheck-regressions",
        "command": "python3 -m unittest discover -s tests -p test_precheck_causes.py",
        "executor": "codex",
        "profile": "deep_coder",
        "context": "project",
        "required": true,
        "criteria": [
          "AC-1",
          "AC-2",
          "AC-3",
          "AC-4"
        ],
        "level": "boundary",
        "provenance": "agent",
        "reason": "redacted failures, retry evidence, real failing Git subprocess and inventory coverage",
        "head_sha": "aef0cf4a6077c7748819bbf8a14b6e6363f9825d",
        "result": "pass",
        "evidence": [
          {
            "exit_code": 0,
            "executed": 11
          }
        ],
        "cause": "",
        "attempts": []
      },
      {
        "id": "manifest-regressions",
        "command": "python3 -m unittest discover -s tests -p test_fast_path.py",
        "executor": "codex",
        "profile": "deep_coder",
        "context": "project",
        "required": true,
        "criteria": [
          "AC-3",
          "AC-4"
        ],
        "level": "test",
        "provenance": "agent",
        "reason": "register the inventory test with fast-path and Windows shard guards",
        "head_sha": "aef0cf4a6077c7748819bbf8a14b6e6363f9825d",
        "result": "pass",
        "evidence": [
          {
            "exit_code": 0,
            "executed": 42
          }
        ],
        "cause": "",
        "attempts": []
      },
      {
        "id": "dispatch-compatibility",
        "command": "python3 -m unittest discover -s tests -p test_dispatch_failures.py",
        "executor": "codex",
        "profile": "deep_coder",
        "context": "project",
        "required": true,
        "criteria": [
          "AC-2",
          "AC-4"
        ],
        "level": "test",
        "provenance": "agent",
        "reason": "audit-unavailable classification without starting an attempt",
        "head_sha": "aef0cf4a6077c7748819bbf8a14b6e6363f9825d",
        "result": "pass",
        "evidence": [
          {
            "exit_code": 0,
            "executed": 30
          }
        ],
        "cause": "",
        "attempts": []
      },
      {
        "id": "consent-regression",
        "command": "python3 -m unittest discover -s tests -p test_consent_variants.py -k test_rev016_bash_baseline_and_real_hook_allow_and_deny",
        "executor": "codex",
        "profile": "deep_coder",
        "context": "project",
        "required": true,
        "criteria": [],
        "level": "test",
        "provenance": "agent",
        "reason": "rerun the time-budget errors without competing full-suite execution",
        "head_sha": "aef0cf4a6077c7748819bbf8a14b6e6363f9825d",
        "result": "pass",
        "evidence": [
          {
            "exit_code": 0,
            "executed": 1
          }
        ],
        "cause": "",
        "attempts": []
      }
    ]
  }
}

Checks of the pushed head were read once under the parallel CI policy; both were queued and remain pending in this report.

{
  "checks": {
    "head_sha": "aef0cf4a6077c7748819bbf8a14b6e6363f9825d",
    "passed": 0,
    "failed": 0,
    "pending": 2,
    "items": [
      {
        "name": "Classify the change",
        "state": "pending"
      },
      {
        "name": "Skills, manifests, and Bash installer",
        "state": "pending"
      }
    ]
  }
}

@datacas

datacas commented Nov 18, 2026

Copy link
Copy Markdown
Owner Author

[CCR-20261118-001] · senior_reviewer · anthropic/claude-opus-5-5→? · high · schema:2

Initial review of #513 at aef0cf4a6077c7748819bbf8a14b6e6363f9825d: REVIEW_PASSED_AWAITING_CI.

Full accumulated diff vs origin/main reviewed (830 lines, 11 files). 0 findings, nothing blocks approval.

  • AC-1 to AC-4 are met. Pre-dispatch causes are redacted and kept. An unreadable ref audit retries once, then stops with publication_unverified / not_started and no launch. The handler inventory is in docs/capability-parity-audit.md. 11 new regression tests.
  • The security audit ran because the plan requires it. No confirmed vulnerability.
  • All 4 configured checks pass locally. CI: 5 passed, 1 pending (Windows test shard (windows-3)).
Review details, verification and receipt

Scope

The runtime supplied the diff: base origin/main, merge base 87675a97fab07dfdecf4265f34d539ea9f511756, 830 lines (line count confirmed), read in sections. Files reviewed:

  • scripts/executors.py, scripts/publication_audit.py and scripts/run_cycle.py;
  • docs/capability-parity-audit.md and docs/role-workspace-policy.md;
  • scripts/fast-path-tests.json and scripts/windows-test-shards.json;
  • tests/test_precheck_causes.py (new), tests/test_executors.py, tests/test_dispatch_failures.py and tests/test_run_cycle.py.

Findings

None. No earlier review findings exist on this change request, so no IDs were recovered.

Acceptance criteria

AC Result Evidence
AC-1 met git grep -n "could not" over the five dispatch files finds no precheck message without a cause. _inspect_remote_refs deliberately keeps a stable, credential-safe category; its inventory row records and justifies this.
AC-2 met _verified_read_only_refs retries discovery and capture once, then raises ReadOnlyAuditUnavailable. dispatch maps it to FAILED / publication_unverified / start_state=not_started, with artifacts.publication_audit.errors kept. A failed post-dispatch audit keeps the stage artifacts. This is covered by test_capture_failure_retries_and_stops_with_redacted_cause, test_real_git_failure_stops_before_executor and test_unreadable_post_audit_preserves_stage_artifacts.
AC-3 met The inventory table is committed. test_every_broad_handler_has_an_inventory_row walks the AST of all five modules and requires an exact `owner
AC-4 met tests/test_precheck_causes.py contains 11 tests, all passing. The updated stop assertions in the existing suites also pass.

Notes (non-blocking, not findings)

  • _git_read now returns redacted stderr in place of stdout when Git exits with a code other than 0 or 1. Every caller (executors.py lines 204, 777, 2043–2065, 3252, 3288) reads the value only on exit 0, or reports it as a diagnostic. No regression.
  • A checkout that is not a Git repository now fails remote discovery first. It is reported as publication_unverified rather than read_only_verification. Both stop before dispatch, and the tests were updated on purpose to match.
  • The inventory test checks coverage in one direction only. It does not detect stale rows left for handlers that were removed.
  • Several new CycleDriverError messages end with a trailing space. This is cosmetic.

Security

The plan has security_required=true, so cc-security-review ran. The deterministic rule did not fire: no path matched the mandatory minimum and the PR has no labels. My own reading also marks the change as sensitive, because untrusted subprocess output enters result artifacts and reports.

No confirmed vulnerability was found. Reviewed surfaces:

  • Redaction: Git/gh stderr and stdout flow into detail and artifacts only through cycle_journal.redact. It removes URL credentials, tokens and control characters, and bounds the text after redaction.
  • Retry: a persistent failure never dispatches. A successful retry dispatches only after a valid before capture.
  • Fail-closed: launch.assert_not_called() and start_state=not_started are asserted.

Residual risk: redaction is pattern-based, so it is not proof that arbitrary provider diagnostics contain no sensitive data.

Verification (head aef0cf4, executed in the disposable review clone)

Check Command Exit Result
configured-test python3 -m unittest discover -s tests 0 Ran 2663 tests in 651.376s, OK (skipped=1)
verify:validate-package python3 scripts/validate-package.py 0 VALID: 16 skills
verify:unittest python3 -m unittest 0 Ran 0 tests, OK
verify:basedpyright basedpyright 0 0 errors, 0 warnings, 0 notes
targeted python3 -m unittest discover -s tests -p test_precheck_causes.py -v 0 11 tests OK

Not verified: no live GitHub or remote outage was reproduced. Provider failures were exercised through injected subprocess results. The Git exit-128 boundary ran against a real local repository.

CI (parallel policy, read once, not waited on)

Check State
Classify the change pass
Skills, manifests, and Bash installer pass
Windows test shard (windows-1) pass
Windows test shard (windows-2) pass
Windows test shard (windows-4) pass
Windows test shard (windows-3) pending

test_precheck_causes is assigned to shard windows-4, which passed.

Documentation

docs/role-workspace-policy.md documents the retry, the stop and the evidence fields. docs/capability-parity-audit.md contains the inventory. The change adds no new veto row.

@datacas
datacas merged commit ddf2838 into main Nov 18, 2026
7 checks passed
@datacas
datacas deleted the fix/510-precheck-real-cause branch November 18, 2026 21:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

A failed precheck or probe must report its real cause and degrade, not become a generic BLOCKED

1 participant