Skip to content

[Framework] Add opt-in skipping of empty associated slices in grouped combinations - #16005

Open
BongHwi wants to merge 3 commits into
AliceO2Group:devfrom
BongHwi:pair-improve-skip-empty-data
Open

BongHwi wants to merge 3 commits into
AliceO2Group:devfrom
BongHwi:pair-improve-skip-empty-data

Conversation

@BongHwi

@BongHwi BongHwi commented Oct 25, 2026

Copy link
Copy Markdown
Contributor

GroupedCombinations currently constructs typed associated slices before the consumer can check whether they are empty. For sparse associations, repeatedly constructing these slices can dominate the mixing workload, even when the consumer immediately discards empty tuples.

This change adds an explicit GroupedSlicePolicy::SkipEmpty option. It checks association counts through the existing sorted SliceCache before constructing typed slices.

  • IncludeEmpty remains the default, preserving the existing yielded stream.
  • Skipped tuples are traversed using the original grouping policy. Categories, mixing depth, partner direction and window boundaries are preserved.
  • Window metadata retains its original-policy meaning.
  • The opt-in accepts raw associated tables and retains the existing sorted-cache and lifetime requirements.

Consumers should enable this option only when processing an empty-associated tuple has no observable effects, including normalization, counters or window bookkeeping.

Performance

A three-run evaluation of an Omega2012 mixed-event workload on macOS arm64, using a fixed seven-DF input and mixing depth 10, gave the following mean CPU times:

Mode Baseline CPU SkipEmpty CPU Reduction
Xi, K0s 235.26 s 5.39 s 97.71%
Xi1530, K 151.54 s 5.63 s 96.28%

These are complete local DPL CPU times (user + system, including child processes). All valid measurements were included. The Xi, K0s baseline had a coefficient of variation of 5.19%.

The input has sparse cascade associations, so these gains are workload-dependent. Dense inputs may benefit less or incur additional cache-lookup overhead.

Validation

  • Added Arrow-backed integration coverage for Pair/Triple policies, default behavior, iterator copies, bindings, filtered grouping, cache refresh and original window metadata.
  • Rebuilt and ran the new integration scenarios together with related ASoA, ASoAHelpers and GroupSlicer coverage: 50 cases passed.
  • Separately validated the current-source Omega2012 opt-ins across twelve full-input executions, covering same-event, mixed-event and combined processing for both modes.
  • All six baseline-versus-opt-in ROOT-output comparisons passed without numerical tolerances.

Filter empty associations before constructing typed slices while preserving
the original grouping policy and windows. Keep IncludeEmpty as the default
and document consumer and cache preconditions.

Add Arrow-backed integration coverage for Pair and Triple policies, iterator
copies, cache refresh, filtered grouping, bindings and window metadata.
Remove the explanatory comments added with the SkipEmpty policy and its
integration coverage. Keep existing comments, license headers and code
unchanged.
Use the canonical ALICE O2 copyright wording in the integration test.
Align the empty eligibility structs with clang-format 20.
@BongHwi

BongHwi commented Oct 25, 2026

Copy link
Copy Markdown
Contributor Author

This improves a lot for the event-mixing when we handle the very rare daughter case, such as v0 and cascade.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

1 participant