Repository navigation
perf(p2p): pipeline block and header requests during sync - #2162
Draft
nullPointerEnjoyer wants to merge 2 commits into
Draft
nullPointerEnjoyer wants to merge 2 commits into
nullPointerEnjoyer wants to merge 2 commits into
Conversation
Keep the peer's send stream continuous during the block download instead of stopping between batches: while blocks are in flight, the next portion of the staged headers is requested as soon as the outstanding count drops below half the request limit, and header requests keep a look-ahead window of up to twice the header list limit staged ahead of the download. Header responses received during the download are anchored at the last processed block, so they legitimately repeat headers that are still in flight or staged; the repeated prefix is skipped before the remaining headers are staged, keeping the look-ahead a single connected chain onto a known or in-flight header. Diverging lists replace the staged window, branching lists keep the window connected up to the branch point. Staged headers that turned out to be already known are filtered out before the mid-flight request, mirroring the batch drain path. The total number of outstanding requested blocks stays within max_request_blocks_count, which is also the serving limit peers enforce on their side.
Pin the pipelining behavior at small protocol limits: the peer's block requests stay within the request limit and free of duplicates while in flight, batch boundaries overlap, repeated and purely overlapping header lists produce no requests, the staged look-ahead is bounded by twice the header list limit, a diverging header list keeps the window connected, the consecutive known-lists grace still applies after a download, the fork download budget defers lists during a download until its retry, and small request limits degrade to lockstep batching as before. The header list request test now expects the look-ahead top-up request that is sent while the first batch is being downloaded.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Keeps the peer's send stream continuous during the block download instead of stopping between batches:
max_request_blocks_count, so the peer starts serving the next batch without waiting for the current one to fully drain. Total outstanding requests stay withinmax_request_blocks_count, which is also the serving limit peers enforce on their side.2 × msg_header_count_limitheaders staged ahead of the download, so header fetch overlaps block fetch. Requests are issued only while the window is below the target and the previous response actually staged new headers, so repeated or purely overlapping lists cannot turn into request loops (the consecutive known-full-lists backpressure is unchanged).Out-of-order and unsolicited block handling, stall detection, the fork download budget, and
wait_for_clock_diffare unchanged.Verification
cargo test -p p2p --lib: 340 passed (330 pre-existing + 10 new/extended), covering: pipelined request continuity, no duplicate/out-of-limit requests, overlap dedup without spin, window bound under divergent zero-overlap lists (capacity cap exercised at exactly the target), reorg-mid-download connectivity, known-lists grace after a download, budget deferral + retry during a download, and lockstep degradation at small request limits../do_checks.sh(fmt / deny / vet / clippy ×2 / codecheck / wasm-docs): pass.cargo test -p chainstate-test-suite: pass.Notes
masterpost-perf(experimental): IBD and indexer performance improvements #2156.ibd-abharness, run D vs run C baseline 5,164 s). Expected effect is largest on high-RTT peers; will be appended when available.