[zzwong]

Summary

Run offline report-identity fixtures through cr benchmark run identity.yml, using either --provider NAME --model ID or --candidate NAME. Strict suite configuration names the adapter, endpoint and credential-store reference; a saved candidate binds a model to its provider. Selection is explicit, with no inferred provider, model or paid fallback.

Support System One-compatible services, including Jev, OpenAI Decisions and Cloudflare Clef through shared prompts, pair preparation, budgets, receipts and deterministic replay. Clef requires explicit acknowledgement that complete-context consumption is unverified. Adapter clients contain no identity prompt or price policy. Existing PR-review suites keep their format and behavior.

The shared executor, selected-store credential resolution and complete serialized-fixture preflight are included from #715, #716, #718 and #719. Retained original records and resolved provider/model provenance are hash-bound; missing prices or uncertain cache usage keep cost unknown. Unshipped model-specific commands are replaced without aliases. This is an offline experiment path; live reviews and posting are unchanged.

Validation

• Full tests, lint, module tidy and build passed on the provider stack. Hermetic command and adapter checks cover direct/saved selection, strict YAML, routing before credential lookup, refusals, cache uncertainty, complete budget rejection and failure retention.
• Compiled CLI checks use zero provider calls to verify resolved provenance, original-record retention, private permissions, manifest hashes and independent replay. Serialized-size boundary tests include decision growth and Unicode/HTML escaping.
• Saved Jev and Decisions prepared-request hashes remain unchanged. Historical synthetic adapter smokes are reused; no new inference calls or full matrix were run. Clef's live transport smoke remains pending.
• The integration includes the current main-branch workspace layout and cleanup behavior, with generic test fixtures.

[zzwong]

Runtime credential resolution bypasses the selected store, retained fixtures can exceed replay's reader limit, and the cohort command test does not verify which records each request contains. Details inline.