Repository navigation
test(shutdown): compare the packet demo's maps in any key order - #9
Merged
Merged
Conversation
The shutdown probe compared the packet demo's output with a fixed string that lists map keys in source order. Nera prints maps of up to 32 keys in atom-table order, so the handler case failed on every run even though the output was complete and correct. The probe now reads those two lines as Erlang terms and compares maps as sets of entries. Every line, key and value must still be there, spacing included; a missing or cut line, a changed value or a repeated key fails. Negative controls run at start-up so the comparison cannot quietly become lenient.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
The opt-in shutdown probe failed on every run of main: its packet demo case expected map keys in source order, but Nera prints maps of up to 32 keys in atom-table order.
What changed
Before and after
Before: On main the probe passed the first four programs, then stopped with "incomplete/unexpected output" on the packet demo, whose output was complete and correct.
After: The probe passes all ten cases (five programs, two scheduler settings) and still fails when the packet demo's output is incomplete or wrong.
Only an opt-in test script and the contributing guide change; nothing a person using Nera sees changes.
Checked
How it was checked
cargo build --release --locked -p nera-runtime, then -p nera: passed
python3 scripts/test-native-shutdown.py at main's version: exit 1 on the handler case
python3 scripts/test-native-shutdown.py at this version: exit 0 on 11 runs (one, then ten in a row), 10 of 10 cases within the 5 ms budget each time
Same probe with demo/serverless/handler.erl temporarily altered three ways (not committed): exit 1 each time, on the handler case
HANDLER_OUTPUT is byte-identical to the previous expected string
python3 scripts/ci/changes.py on the changed files: every CI job runs, since the probe script has no rule of its own
Risk
Risk above low: Novelty, medium — A small new reader for printed Erlang maps, tuples, atoms and integers
Full risk assessment