[ddeboer]

Once AdaptiveTimeoutPolicy tightened an endpoint, it stayed tightened for as long as requests kept timing out, even when they were timing out at the tightened budget. On a healthy endpoint whose heavy queries take longer than tightenedMs, every heavy stage then failed until some cheap query happened to succeed. In the run in the issue, that took out ten stages.

The rule now: a timeout at the tightened budget doesn’t count toward tightening, because it says nothing about whether the endpoint is healthy. The policy re-probes instead, giving the next request on that endpoint defaultMs. If the probe succeeds, the endpoint relaxes. If it times out as well, the endpoint stays tightened, and the next timeout at the tightened budget schedules the next probe. Only one probe is handed out at a time; concurrent requests keep the tightened budget.

What this costs and buys:

• A healthy-but-slow endpoint loses one request after tightening, not a run of stages. In the issue’s run, triples.rq would still time out at 10 s, but class-partition.rq would get the full 5 minutes and relax the endpoint.
• An endpoint that keeps timing out at defaultMs now waits out the full budget on every other request rather than never again. That makes it slower to fast-fail than before, but it still takes about half the time it would without the policy.

To know which budget a timed-out request ran under, afterRequest now receives timeoutMs, the budget beforeRequest returned for that attempt. With requests in flight at the same time, the policy can’t reliably work that out from its own state or from durationMs (an upstream 504 can come back fast). That makes timeoutMs a required field of AfterRequestContext, hence the !.

The reference and the how-to guide describe the new rule.

Fixes #513