Skip to content

feat(svar2): flat "variants" output with ride-along flank_tokens - #428

Merged
d-laub merged 1 commit into
mainfrom
feat/svar2-flat-variants
Nov 18, 2026
Merged

d-laub merged 1 commit into
mainfrom
feat/svar2-flat-variants

Conversation

@d-laub

@d-laub d-laub commented Nov 18, 2026

Copy link
Copy Markdown
Collaborator

Closes #427

Problem

On a .svar2-backed dataset, with_seqs("variants").with_output_format("flat") returned RaggedVariants and never raised, so the ride-along flank_tokens were never produced.

Design

  • Svar2Haps.__call__ now honors flat for variants. A new _decode_variants holds the existing read-bound decode and contig-group stitching, and the ragged and flat outputs both wrap its result.
  • The flat path runs the variants-mode assemble_variant_buffers kernel that .svar's get_variants_flat uses, over the decoded arrays through an identity gather. It returns FlatVariants (alt, start, ilen when requested, store INFO/FORMAT fields) plus flank_tokens. No Rust changes.
  • Bare-ALT convention and the flank span: on .svar2 a pure deletion's alt is b"" (.svar keeps the anchor). start/ilen are identical on both backends because the deletion starts at the anchor, so the flank span [start, start - min(ilen,0) + 1) and therefore flank_tokens are byte-identical. Unlike alt_window, no re-anchoring is needed. Flank tokens stay reference-oriented on minus strands. alt is reverse-complemented by the caller, as for .svar.
  • Also fixed: ragged .svar2 variants output ignored dummy_variant without raising. It now fills empty groups the way .svar does.

Supported / raises

  • Supported, flat parity with .svar: phased, unphased_union, dummy_variant, var_fields (with and without ilen, plus store INFO/FORMAT fields), minus strand, multi-contig, var_filter="exonic", and to_dataloader(mode="buffered" | "double_buffered").
  • Unchanged restrictions: max_jitter>0 or jitter>0 still raises NotImplementedError (right-clip). Spliced variants still require ragged output (ValueError, same as .svar).

Tests

  • The strict xfail test_svar2_flank_tokens_match_svar1 is now a regular test.
  • New tests: parity between .svar and .svar2 flat output (12-way parametrization: union × dummy × var_fields, on a mixed-strand bed), minus-strand RC, multi-contig, store fields vs ragged, buffered dataloader, the jitter guard, and ragged dummy_variant parity.
  • Before the fix, 18 of the new or converted tests fail. After it, all pass.
  • Full pixi run -e dev test: 1352 passed, 52 skipped, 4 xfailed. cargo test passes. ruff, ruff format, pyrefly, and the docstring check are clean.

Perf

1kg test data (100 × 10 kb regions, 5 samples, about 14.8k variants, L=16): .svar2 flat output takes 4.8 ms against 1.6 ms for .svar. .svar2 ragged without flanks takes 3.5 ms. On rows where both backends return the same variants, flank_tokens are identical. The 6 of 1000 rows that differ come from variants .svar includes up to 55 bp past the region end. That difference already shows up in ragged output, so it is not related to this PR.

Docs

faq.md, format.md (svar2 supported list and pure-deletion section), and skills/genvarloader/SKILL.md.

🤖 Generated with Claude Code

https://claude.ai/code/session_01X11f2DWnh5PcaTMo2g6JQT

On a .svar2-backed dataset, with_seqs("variants").with_output_format("flat")
silently returned RaggedVariants, so flank_tokens were never produced. Svar2Haps
now honors `flat`: it decodes the block once (shared `_decode_variants`, the same
read-bound decode the ragged path uses) and runs the shared variants-mode
assemble_variant_buffers kernel over the decoded arrays via an identity gather,
returning FlatVariants with flank_tokens, as SVAR1's get_variants_flat does.

Flat output matches .svar field for field (alt, start, ilen when requested,
store INFO/FORMAT fields, flank_tokens) except the documented bare-ALT
convention: a pure deletion's alt is b"" on .svar2. start/ilen are identical
(the deletion starts at the anchor), so the flank span and flank_tokens are
byte-identical without re-anchoring. Flank tokens stay reference-oriented on
negative strands; alt is RC'd by the caller, as for .svar.

Ragged svar2 "variants" output also silently ignored dummy_variant; it now
fills empty groups like .svar (routed through the flat fill + to_ragged).

The strict xfail test_svar2_flank_tokens_match_svar1 is now a real test, plus
.svar vs .svar2 flat parity tests over phased/unphased_union, dummy_variant,
var_fields, minus strand, multi-contig, store fields, the buffered dataloader,
and the max_jitter>0 guard.

Closes #427

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01X11f2DWnh5PcaTMo2g6JQT
@d-laub
d-laub marked this pull request as ready for review November 18, 2026 17:42
@d-laub
d-laub merged commit 2053ec8 into main Nov 18, 2026
7 of 8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

svar2: flat "variants" output silently returns RaggedVariants, so flank_tokens are never produced

1 participant