Repository navigation
Conversation
…lease dates Cards for OpenAI's GPT-6.2 Astra and GPT-6.2 Sol, released 2026-11-17, written before their onboarding. The change touches no published artifact. With no --model, eval-no-tools now sends requests to both models along with the rest of config.MODELS. - Route: both ids join GPT_6_MODELS, so they take the Responses-API transport, the local litellm registration and the Responses pricing branch. OpenAI's API changelog says tool calling on these models requires the Responses API. - Price: Astra $8 / $40 per 1M input/output tokens and $0.40 cached input; Sol $2 / $10 and $0.10 (OpenAI's model and pricing pages, read 2026-11-17). Cache reads are a twentieth of input, as on the GPT-6.1 rows, so each override carries its cache_read rate. Cache writes at the 30-minute rate are 1.25x input, the harness default. The listed long-context rates (double input and cached input, 1.5x output above 272k input tokens) are the structure the harness already applies to every Responses row. - Cards: written from OpenAI's documentation before any request to either model, and their notes say so. They take the GPT-6.1 Sol and Terra treatment: the forced answer tool, thinking_budget=True (the 16,384-token starting budget and 300s timeout) and no reasoning parameter. OpenAI's pages state no default reasoning effort and do not say whether a forced tool_choice works on these models; the onboarding gauntlet settles both and supplies the expected cost per scenario. The module docstring now says a card written before its model's onboarding says so in its notes. - Release dates: 2026-11-17 for both, from OpenAI's API changelog and its Developer Community announcement, in paper_results and the app's mirror. - Tests: the cards' treatments, cost reconstruction at the listed standard and long-context rates, the request shape (forced tool; no reasoning, temperature, top_p or logprobs), the listed cache rates and litellm entries, and the release dates. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The check found nothing above a nit. This takes four of its notes: Sol's long-context comment no longer says more than the pricing page does, the forum post is called a forum announcement, a test docstring stops claiming that the models reason by default, and one docstring is reflowed. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
This branch was successfully deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
OpenAI released GPT-6.2 Astra (
gpt-6.2-astra) and GPT-6.2 Sol (gpt-6.2-sol) on 2026-11-17. This adds both to the harness so the board can run them:Status: draft, written before onboarding
Both cards were written from OpenAI's documentation and record no request to either model. Their notes say so. The onboarding gauntlet settles what the documentation doesn't state:
tool_choiceworks on these models;The cards' notes and comments get rewritten in the "Onboarded" form once the gauntlet has run. This stays a draft until then.
Test plan
tests/test_model_cards.py,tests/test_eval_no_tools.py,tests/test_onboard.py,tests/test_supervisor.py,tests/test_reasoning_effort_override.py,tests/test_cli_helpers.py,tests/test_cost_latency.pyandtests/test_disclosures.pypass locally, one file at a time.tests/test_paper_results.pypass (-k release_date).tests/test_paper_results.pyandtests/test_notes.pyin full: not run locally to the end (the host had a 2 GB per-run memory limit today), so CI is the full run.ruff checkandruff format --check.🤖 Generated with Claude Code