Skip to content

feat(benchmark): add Cloudflare Clef benchmark providers - #716

Merged
zzwong merged 7 commits into
zzwong/offline-openai-decisions-identityfrom
zzwong/offline-clef-identity
Nov 10, 2026
Merged

zzwong merged 7 commits into
zzwong/offline-openai-decisions-identityfrom
zzwong/offline-clef-identity

Conversation

@zzwong

@zzwong zzwong commented Nov 7, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Add the cloudflare_clef provider type to report-identity suites, with explicit clef or clef-flash selection. cr benchmark run resolves the configured Cloudflare account and credential store while sharing the identity prompt, System One codec, execution runner and replay.

Cloudflare can truncate long state without a completeness receipt. Runs require --allow-unverified-context and record context_status: unverified; explicit truncation signals fail. A parsed choice does not establish complete input consumption.

Includes shared execution and credential and replay contract fixes. The unshipped infer-clef command is removed without an alias. Live review behavior is unchanged.

Validation

  • Full tests, lint, module tidy and build passed. Provider, command and replay regressions use injected transports and local fixtures.
  • Compiled CLI checks cover direct and saved candidates, empty inputs, whole-budget rejection, missing acknowledgement and invalid selectors without provider calls. They verify fixed routes, Bearer provenance, unverified context, retained originals, independent replay and private artifacts.
  • Tests cover both selectors, account and candidate validation, explicit truncation, usage metadata and response bounds. Shared contract tests cover selected-store credentials and complete fixture-size boundaries.

Live Cloudflare smoke remains pending because no scoped credential is configured. The preserved local evidence does not establish complete-context model quality.

Reuse the System One codec for Clef and Clef-flash requests while validating Cloudflare envelopes, routing and usage. Mark provider context completeness as unverified.
Replace only the usage value so valid response metadata cannot inflate past the wire size guard. Retain known usage and the original response digest for ASCII and Unicode metadata.
Reuse the shared identity question and replay runner for Cloudflare models. Require an explicit context acknowledgement and retain unverified context, usage and complete original records in private receipts.
@zzwong zzwong changed the title feat(benchmark): add Cloudflare Clef identity inference feat(benchmark): add Cloudflare Clef benchmark providers Nov 7, 2026

@zzwong zzwong left a comment •

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The unverified-context acknowledgement and pending Cloudflare smoke are documented accurately. Credential and retention corrections are tracked in #719.

Share identity preparation, inference, receipts and private artifact publication across benchmark providers. Enforce selected-store credentials, complete replayable fixtures and bounded execution before provider calls.
@zzwong
zzwong marked this pull request as ready for review November 10, 2026 14:39
@zzwong
zzwong merged commit 5222120 into zzwong/offline-openai-decisions-identity Nov 10, 2026
12 checks passed
@zzwong
zzwong deleted the zzwong/offline-clef-identity branch November 10, 2026 14:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant