[itsrayeedscodespace]

Fixes #3219.

Root cause

WebhookSender::should_send returned true for every Output event before the throttle check. Its comment said output was effectively unthrottled in the old Python runtime. It wasn't: ResponseThrottler throttled every non-terminal event, and the Go runtime's SendConditional throttled logs and output at 500ms. docs/http.md and COG_THROTTLE_RESPONSE_INTERVAL in docs/environment.md also promise that output and logs webhooks go out at most once per interval.

Every webhook is a full snapshot, with all logs and all outputs so far. One webhook per chunk therefore means O(chunks × logs) bytes. Prediction::fire_webhook also built that snapshot before should_send ran, so throttled and filtered events still cloned all logs and outputs.

Change

• should_send no longer exempts Output. output and logs share the throttle again.
• fire_webhook checks should_send before building the payload. should_send is now pub(crate); send keeps its signature and still checks.

The terminal path is unchanged. Terminal webhooks are never throttled and always carry the full state, so a dropped progress webhook loses no data.

Before / after

Rust harness. This drives a real Prediction and WebhookSender against a local axum receiver that records count, bytes, peak in-flight requests and arrival order. 1,000 chunks 5ms apart, 500ms throttle, debug build. The harness wasn't committed.

Scenario	main	this PR
1 MB log, fast receiver	1,002 webhooks, 1.00 GB	15 webhooks, 14 MB
100 B log, fast receiver	1,002, 3.7 MB	15, 57 KB
1 MB log, 200ms receiver, peak in flight	8	1
1 MB log, failed after 500 chunks	502, 502 MB	8, 7 MB
1 MB log, canceled after 500 chunks	502, 502 MB	8, 7 MB
1 MB log, filter start,completed: time spent in Prediction calls	245ms	9ms
1 MB log, all events: time spent in Prediction calls (snapshot + serialization)	20.2s	0.72s

In every run, the terminal webhook arrived last and carried all outputs and the full logs.

Issue repro in Docker. I ran the issue's predict.py and sink with locally built coglet wheels, linux/amd64 under emulation on Apple Silicon. I checked the _impl.abi3.so hash in each image against the wheel I built.

Build	log_bytes	Webhooks	Total bytes
main	1,000,000	630	631 MB
main, COG_THROTTLE_RESPONSE_INTERVAL=5	1,000,000	632	633 MB
main	100	654	2.2 MB
this PR	1,000,000	3	2.0 MB
this PR	100	3	7 KB

Under emulation the chunks reach coglet in one burst at the end of the prediction, on main as well. So this setup shows the count reduction but not the 500ms spacing; the Rust harness covers the spacing.

Relationship to #3211

#3211 sends progress webhooks one at a time from a single task per prediction and replaces a waiting request with a newer one. That bounds in-flight requests and fixes ordering. It doesn't throttle: a fast receiver drains the queue as fast as chunks arrive. The payload is also still built and serialized for every chunk. The harness on #3211 alone gives 1,002 webhooks and 1.0 GB with a fast receiver, and 19.8s of payload building.

The two changes are complementary. I applied this patch on top of #3211 and all 241 Rust tests passed. The conflicts are only in tests:

• Both PRs edit send_output_not_throttled; this PR renames it to send_output_throttled.
• Both add tests at the same place in prediction.rs.
• On Send single outputs only in the final webhook, and send webhooks in order #3211, the new prediction test needs pred.mark_streaming().

With both applied, peak in-flight stays at 1 even when the receiver hangs for 5s.

Tests

• prediction::tests::output_webhooks_are_throttled_and_terminal_has_everything: 100 KB of logs, 100 chunks, then succeeded, failed or canceled. Checks that only the start and terminal webhooks are sent, and that the terminal one has all 100 outputs and the full logs. On main it gets 102 webhooks.
• webhook::tests::send_output_throttled replaces send_output_not_throttled.
• webhook::tests::should_send_throttles_output_and_logs_together and should_send_respects_filter.

All three new behavior tests fail with the Output exemption put back. I ran the new tests 30 times with no failures.

mise run test:rust   # 233 tests run: 233 passed, 0 skipped
mise run fmt:rust    # clean
mise run lint:rust   # clean
cargo clippy -p coglet --all-targets -- -D warnings
                     # only `items after a test module` in lib.rs, which main has too

Behavior changes and limits

• An output event within the throttle interval of the last webhook isn't sent. That includes the start webhook, so the first output webhook can be up to 500ms later than on main. The Python and Go runtimes behaved the same way.
• The throttle has no trailing edge, as with logs today. If a model yields and then pauses, the latest outputs arrive with the next webhook or the terminal one. A trailing edge fits naturally in Send single outputs only in the final webhook, and send webhooks in order #3211's queue.
• Without Send single outputs only in the final webhook, and send webhooks in order #3211, a receiver that hangs still has one new request in flight per interval per prediction: 10 in my 5s-hang run, at most about 60 at the 30s client timeout. On main it was chunks per second × latency. Send single outputs only in the final webhook, and send webhooks in order #3211 brings it to 1.
• Non-terminal payloads still carry the full logs, so each one is as large as the logs. The issue's suggestion to cap logs in non-terminal payloads would change the payload contract, so I left it for maintainers to decide.

Verified and Tested with Claude Code