Skip to content

Reduce bit-vector append overhead - #13

Open
adamant-pwn wants to merge 1 commit into
jermp:mainfrom
ratschlab:upstream/append-bits
Open

adamant-pwn wants to merge 1 commit into
jermp:mainfrom
ratschlab:upstream/append-bits

Conversation

@adamant-pwn

Copy link
Copy Markdown
Contributor

Bulk bit-vector construction repeatedly appends packed words. Share the growth path for aligned and overflowing appends, remove the cached last-word pointer, and use unsigned shift counts. Assigning the packed value after emplace_back() avoids passing it by reference into vector growth, removing spills in the measured GCC loop. Existing builder calls require no changes.

In SSHash's encode_strings, median CPU-cycle ratios versus the original append implementation were:

Compiler DNA, 2047 chars Protein, 2047 chars DNA, 63 chars Protein, 63 chars
GCC 15.3 0.813 0.869 0.848 0.892
Clang 23.1 0.907 0.944 0.815 0.965

Measured on a Ryzen 9 7945HX with -O3 -DNDEBUG -mavx2 -mbmi2, using seven randomized paired blocks per workload and at least 0.55 seconds per sample. Long inputs contained 1,800 records; short inputs contained 60,000. Timing includes stream parsing and encoding, with file I/O and packed-output checks outside the measured section.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant