Skip to content

refactor(benchmark): share identity experiment execution - #718

Merged
zzwong merged 2 commits into
zzwong/offline-clef-identityfrom
zzwong/shared-identity-experiment-core
Nov 10, 2026
Merged

zzwong merged 2 commits into
zzwong/offline-clef-identityfrom
zzwong/shared-identity-experiment-core

Conversation

@zzwong

@zzwong zzwong commented Nov 9, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Share report-identity preparation, sequential inference, receipts and private artifact publication between offline benchmarks and the separate diagnostic observer. System One request construction is shared; OpenAI Decisions and Clef retain their provider-specific adapters.

Direct and saved-candidate runs resolve keys through the configured credential store and provider-specific key mapping. Output reservation and complete-fixture size checks happen before credential lookup or provider calls. Retained inputs preserve complete report text and replayable request encoding.

Includes the credential and replay contract fixes. Post-review observer wiring remains in the separate shadow feature; this change does not alter live review verdicts or posting.

Validation

  • make check passed, including tidy, formatting, lint, the full test suite and build.
  • Hermetic command tests cover configured stores, direct and saved candidates, missing or conflicting credentials, complete request contents and zero-call rejection paths.
  • Boundary regressions independently encode the expected fixture, accept the exact size limit and reject one byte over before credentials or inference. Provider accounting preserves unknown cache-write costs.
  • Validation uses local fixtures and mock providers.

@zzwong zzwong left a comment •

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The shared execution path preserves request and receipt behavior. Reservation failures stop before credential lookup or provider calls.

Resolve provider credentials through the selected secret store and preserve complete replayable inputs. Cover command credential selection, fixture size boundaries and unknown cache-write costs.
@zzwong
zzwong marked this pull request as ready for review November 10, 2026 14:34
@zzwong
zzwong merged commit 22cebe6 into zzwong/offline-clef-identity Nov 10, 2026
12 checks passed
@zzwong
zzwong deleted the zzwong/shared-identity-experiment-core branch November 10, 2026 14:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant