Skip to content

fix(api): price spans for models tokencost does not list instead of booking $0 - #1489

Open
Mattbusel wants to merge 1 commit into
AgentOps-AI:mainfrom
Mattbusel:fix/price-models-missing-from-tokencost
Open

Mattbusel wants to merge 1 commit into
AgentOps-AI:mainfrom
Mattbusel:fix/price-models-missing-from-tokencost

Conversation

@Mattbusel

Copy link
Copy Markdown

SpanMetricsMixin._calculate_cost books a span at $0 whenever tokencost raises for its model (span_metrics.py):

try:
    completion_cost = costs.calculate_cost_by_tokens(tokens, self.model_for_cost, direction)
except Exception:
    return Decimal(0)

tokencost's latest release is 0.1.26 (2025-08-13), and its table has no gpt-5.2, gpt-5.4, gpt-5.5, gpt-6, Claude Opus 4.6+ or Sonnet 4.6, or Gemini 3.x rows. calculate_cost_by_tokens raises KeyError for each of them, so every span on those models shows $0 prompt, completion and total cost. A gpt-5.5 call with a 100K-token prompt and 5K output costs $0.65 and is shown as $0.

What this changes

  • When tokencost has no row, the cost comes from llm-prices (dated list prices with the provider page each one came from, no dependencies). tokencost still wins when it has the model.
  • A model neither table knows still costs 0, as before.
  • llm-prices>=1.8.0 is added to app/api/pyproject.toml and uv.lock.

Tests

app/api/tests/test_span_metrics_cost.py: gpt-5.5 input and output priced ($0.50 for 100K input, $0.15 for 5K output), claude-opus-4-7 priced, gpt-4o still priced by tokencost, unknown model and missing model still 0. 5 passed. ruff format and ruff check are clean.

The collector's cost table (app/opentelemetry-collector/builder/costs) is also built from tokencost and has the same gap; happy to follow up there if this direction works for you.

Disclosure: I maintain llm-prices.

…ooking $0

SpanMetricsMixin._calculate_cost returned Decimal(0) whenever tokencost
raised for a model, and tokencost's last release (0.1.26, Aug 2025) has
no row for gpt-5.2/5.4/5.5, the gpt-6 family, Claude Opus 4.6+ or
Gemini 3.x. Spans on those models showed $0 in prompt, completion and
total cost.

When tokencost has no row, the cost now comes from llm-prices (dated,
source-cited list prices, no dependencies). tokencost still wins when it
has the model, and a model neither table knows still costs 0.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant