Repository navigation
feat(review): add opt-in post-review identity shadow - #720
Merged
Merged
Conversation
Add an explicit provider command for caller-owned cohort and prior-thread fixtures, with all-pair preflight and retained replay artifacts. Keep provider access isolated from live reviews and record partial failures, usage, and cost estimates without credentials.
Separate the reusable choice client from report identity prompts and make model, endpoint, authentication, limits and prices explicit caller settings. Preserve private replay artifacts and distinguish unknown costs from known zero.
Exercise TypeSafe authentication and resolved model handling with the shared identity adapter. Add small synthetic cohort and prior-thread fixtures for provider smoke checks.
Validate named choice answers and cache usage through a dedicated Decisions transport, retaining safe metadata on refusals and failed requests.
Reuse the System One codec for Clef and Clef-flash requests while validating Cloudflare envelopes, routing and usage. Mark provider context completeness as unverified.
Replace only the usage value so valid response metadata cannot inflate past the wire size guard. Retain known usage and the original response digest for ASCII and Unicode metadata.
Use one versioned report projection and replay runner for System One and Decisions. Preserve request provenance, retained originals and failure accounting while keeping cache-dependent costs explicitly unknown.
…ffline-clef-identity
Reuse the shared identity question and replay runner for Cloudflare models. Require an explicit context acknowledgement and retain unverified context, usage and complete original records in private receipts.
…g/mammoth-identity-shadow
zzwong
commented
Nov 10, 2026
Contributor
Author
There was a problem hiding this comment.
An inconsistent decision cycle can mark the diagnostic complete. Add regression coverage for aggregate-byte limits, the inference deadline and exact prior-pair request contents. The status defect affects diagnostic completeness without changing the authoritative verdict or posting.
Base automatically changed from
zzwong/identity-credential-replay-contracts
to
zzwong/shared-identity-experiment-core
November 10, 2026 14:24
Base automatically changed from
zzwong/shared-identity-experiment-core
to
zzwong/offline-clef-identity
November 10, 2026 14:35
Base automatically changed from
zzwong/offline-clef-identity
to
zzwong/offline-openai-decisions-identity
November 10, 2026 14:41
Add Decisions request and answer handling to provider-driven identity experiments while preserving refusals, cache uncertainty and original evidence. Include shared provider execution, selected-store credentials and replay contract enforcement.
Base automatically changed from
zzwong/offline-openai-decisions-identity
to
zzwong/offline-mammoth-identity
November 10, 2026 14:57
…ammoth-identity-shadow # Conflicts: # docs/architecture.md # internal/architecture/report_identity_test.go
…ty-shadow # Conflicts: # docs/architecture.md # internal/architecture/report_identity_test.go
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Builds on the provider foundation in #714. Add
cr review --identity-shadow --identity-shadow-config <path>to compare an explicitly configured System One model with the review's existing finding groups and prior-thread matching. The diagnostic runs after a newly completed review, its posting lock release, and completion-hook dispatch. It is off by default; a config path alone does nothing.Current findings, verdicts, thread actions, posting, ledger records, fail-on behavior and normal output remain authoritative. The observer receives copied values and returns no decision to the review path. Resumed, retried, aborted and incomplete live reviews skip it. Dry-run opt-in can send diagnostic inputs to the configured inference provider while retaining the usual prohibition on Git-provider writes.
The implementation reuses the offline identity prompts, request preparation, provider adapter, receipts and replay machinery. Complete findings and eligible normalized prior opening bodies are retained in private run-owned artifacts, together with the observed baseline, bounded inference receipts and hash-bound comparisons. Baseline agreement measures agreement with current behavior; it is not an accuracy label or evidence of model qualification.
Limits and operation
The standalone strict YAML requires an HTTPS endpoint, model, exact expected response model, auth scheme and credential-store reference. Both complete pair universes are preflighted and output directories reserved before credentials or HTTP. The default qualification envelope is 40 calls, 327,680 aggregate request bytes, 6,144 state bytes, 8,192 request bytes, 10 seconds per call and a 30-second inference phase; oversized runs are refused rather than sampled. There are no automatic retries or profile/init defaults.
Credential lookup precedes the inference timer and retains the selected backend's existing synchronous bound. It can therefore add command latency after the authoritative review has completed. Local publication adds further time. Unknown prices or usage remain unknown; these limits do not promise a hard dollar ceiling. Ordinary diagnostic errors produce private receipts and sanitized stderr warnings without changing the completed review.
Validation
make checkpassed: tidy, formatting, zero lint issues, the complete Go test suite and build. Race checks passed forinternal/identityexperiment,internal/identityshadowandinternal/app.This feature does not enable the diagnostic on live reviews or change any model-selection or posting policy.