Skip to content

Add the GPT-6.2 Astra and GPT-6.2 Sol model cards and metadata - #268

Draft
MaxGhenis wants to merge 2 commits into
mainfrom
gpt-6.2-cards
Draft

MaxGhenis wants to merge 2 commits into
mainfrom
gpt-6.2-cards

Conversation

@MaxGhenis

Copy link
Copy Markdown
Contributor

Summary

OpenAI released GPT-6.2 Astra (gpt-6.2-astra) and GPT-6.2 Sol (gpt-6.2-sol) on 2026-11-17. This adds both to the harness so the board can run them:

  • Model cards for both, with the GPT-6.1 Sol and GPT-6.1 Terra treatment: the board's forced answer tool on the Responses API and the thinking class's 16,384-token starting budget and 300 s timeout.
  • List prices at OpenAI's Standard tier, read on the release day: Astra $8 input, $0.40 cached input and $40 output per 1M tokens; Sol $2, $0.10 and $10. Above 272k input tokens the existing long-context rule gives OpenAI's listed rates for both.
  • Release dates (2026-11-17) in the paper registry and the app's model metadata.
  • Tests for the cards' treatment, the request shape, the prices at both context tiers, and the release dates.

Status: draft, written before onboarding

Both cards were written from OpenAI's documentation and record no request to either model. Their notes say so. The onboarding gauntlet settles what the documentation doesn't state:

  • whether a forced tool_choice works on these models;
  • the default reasoning effort, and whether the models reason on the board's request;
  • the expected cost per scenario, the timeout and the starting budget;
  • any dated snapshot id.

The cards' notes and comments get rewritten in the "Onboarded" form once the gauntlet has run. This stays a draft until then.

Test plan

  • tests/test_model_cards.py, tests/test_eval_no_tools.py, tests/test_onboard.py, tests/test_supervisor.py, tests/test_reasoning_effort_override.py, tests/test_cli_helpers.py, tests/test_cost_latency.py and tests/test_disclosures.py pass locally, one file at a time.
  • The new release-date tests in tests/test_paper_results.py pass (-k release_date).
  • tests/test_paper_results.py and tests/test_notes.py in full: not run locally to the end (the host had a 2 GB per-run memory limit today), so CI is the full run.
  • ruff check and ruff format --check.
  • Onboarding gauntlet for both models, then the cards' rewrite.
  • Independent review.

🤖 Generated with Claude Code

MaxGhenis and others added 2 commits November 17, 2026 14:37
…lease dates

Cards for OpenAI's GPT-6.2 Astra and GPT-6.2 Sol, released 2026-11-17, written
before their onboarding. The change touches no published artifact. With no
--model, eval-no-tools now sends requests to both models along with the rest
of config.MODELS.

- Route: both ids join GPT_6_MODELS, so they take the Responses-API
  transport, the local litellm registration and the Responses pricing
  branch. OpenAI's API changelog says tool calling on these models requires
  the Responses API.
- Price: Astra $8 / $40 per 1M input/output tokens and $0.40 cached input;
  Sol $2 / $10 and $0.10 (OpenAI's model and pricing pages, read
  2026-11-17). Cache reads are a twentieth of input, as on the GPT-6.1 rows,
  so each override carries its cache_read rate. Cache writes at the
  30-minute rate are 1.25x input, the harness default. The listed
  long-context rates (double input and cached input, 1.5x output above 272k
  input tokens) are the structure the harness already applies to every
  Responses row.
- Cards: written from OpenAI's documentation before any request to either
  model, and their notes say so. They take the GPT-6.1 Sol and Terra
  treatment: the forced answer tool, thinking_budget=True (the 16,384-token
  starting budget and 300s timeout) and no reasoning parameter. OpenAI's
  pages state no default reasoning effort and do not say whether a forced
  tool_choice works on these models; the onboarding gauntlet settles both
  and supplies the expected cost per scenario. The module docstring now says
  a card written before its model's onboarding says so in its notes.
- Release dates: 2026-11-17 for both, from OpenAI's API changelog and its
  Developer Community announcement, in paper_results and the app's mirror.
- Tests: the cards' treatments, cost reconstruction at the listed standard
  and long-context rates, the request shape (forced tool; no reasoning,
  temperature, top_p or logprobs), the listed cache rates and litellm
  entries, and the release dates.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The check found nothing above a nit. This takes four of its notes: Sol's
long-context comment no longer says more than the pricing page does, the
forum post is called a forum announcement, a test docstring stops claiming
that the models reason by default, and one docstring is reflowed.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
@vercel

vercel Bot commented Nov 17, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
policybench-site Ready Ready Preview Nov 17, 2026 8:01pm UTC

Request Review

This branch was successfully deployed

1 active deployment
Preview — 26132a20 Deployed Nov 17, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant