Repository navigation
Conversation
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013VUubhHeYSMGSSDXZkKZss
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AKYzhrMnPK61XHWEch4a6E
…failing list SimSweep: sweep_result() (the log lines and message both games' sweeps print), run_sweep() over SweepVariants, sweep_clean_first() and variant_failure(). SimSeeds: run_seeds() with uint32 seeds, 30 by default, logging mean and worst per measure. KnownFailing: the pass-while-failing wrapper and stale() for entries that name no test. Unit tests for each; adoption in BDSP and FRLG follows per game.
Pokemon Navigation: shared fault sweep runner, seeded rows and known-failing list
…dopt # Conflicts: # SerialPrograms/Source/ComputerPrograms/ComputerPrograms_UnitTestRunner.cpp # SerialPrograms/cmake/SourceFiles.cmake
The sweeps build SweepVariants for Pokemon::run_sweep(), the wild battles run on run_seeds() (30 seeds, as before), the clean-first rule is sweep_clean_first() and the known-failing list is a Pokemon::KnownFailing, with a new test that every entry names a fault sweep test. Test names and the eight known-failing entries are unchanged.
… 47763e7) Conflict in PokemonFRLG_AutoStoryFaultSweep_Tests.cpp sweep_mount_lost: keep seam4-adopt's Pokemon::sweep_result call with stage20-league's variant count of 1.
make_UNIT_TESTS_ALL() passed the code-health function length limit (50 lines) once the SimSweep, SimSeeds and KnownFailing suites were added.
dayfine
left a comment
There was a problem hiding this comment.
Reviewed SHA: fa68899
Verdict
APPROVED
LGTM. No correctness findings; 4 minor and 3 nits below, none blocking. Scope: this PR's own commits (39e9c1a on PokemonFRLG_AutoStoryFaultSweep_Tests.cpp, fa68899 on ComputerPrograms_UnitTestRunner.cpp) and how they use the shared runner. The runner files themselves (Pokemon_SimSweep/SimSeeds/KnownFailing*) come in through the nav/leg-trace merge and were reviewed in #513; I read them only to check the calls.
Correctness
The sweep tests the same thing as before. Compared the Fork Draft CI logs of the base tip (78a0edc, run 43745607030) and this head (run 43770979623):
- Fault sweep tests: 920 on base, 921 on head. The names are identical; the one added is
Fault sweep: every known failing entry names a test. Fault sweep <row> <class>: N variants, M fail.lines: 621 on each side, identical after sorting (same variant counts and fail counts for every row and class).- Failure table rows (
| row | step | class | variant | stopped |): 15 on each side, identical. Known failing until ...lines: 8 on each side, identical.- Wild battles: 88 rows on each side, with the same battle count and clean time per row, and the same pass counts (87 at 30/30, 02.02 at 28/30).
- Whole suite: 35,600 passed, 0 failed (base: 35,581 passed, 0 failed).
By reading:
differs()andline_with_battle()now compareto_string()of the positions instead of the positions.to_string(MapPosition)prints the map and the tile, so the two comparisons agree.- Variants are now built first and run inside
run_sweep(). The lambdas capturerowandcleanby reference; in every sweep both outlive thesweep_variants()call, includingsweep_mount_lost()'s own ridingclean. - Variant counts are
variants.size()and match the old hand counts (DARK_CAVE_SEEDS + 1,warps.size(),moved_step_taps,bike_legs). sweep_clean_first()logs the same "not swept, the row fails clean" line;KnownFailing::run()keeps the catch ofExceptionand the same three outcomes.- Nothing left half-moved in this file: no local
SweepFailure,sweep_result,mean_of,GAME_ODDS_SEEDSorKNOWN_FAILINGmap remains.
One visible change, logs only: the wild-battle summary is now two lines (Seeds <row> wild battles: passed P/N; <measure> mean, worst; ... from run_seeds(), then Fault sweep <row> wild battles: N battles, clean time T s.) instead of one. The means match the old ones (02.02: 146.6 s, 38.2 tiles, 6.7 s per battle). I found no script on this branch or on agents that parses the old line.
CI on the head
Passed: Fork Draft CI (Ubuntu GCC build + unit tests), C++ Code Health, Check for Tabs, C++ CI Python Bindings, Runtime map reader, Story data codec. Skipped because the PR is a draft: the six C++ CI Serial Programs workflows (Mac, Mac Intel, Ubuntu Default, Ubuntu clang, Windows Clang, Windows Default), so MSVC and the clang builds have not compiled this head. Local code health against origin/frlg/stage20-league: 0 failures.
Minor
-
/*param=*/comments on new calls (code healthparam_comment),PokemonFRLG_AutoStoryFaultSweep_Tests.cpp:- line 443,
run_row(row, script, {}, clean.scripted)inwhole_row_variant() - line 651, the same call in the warp no-op variant
- line 689,
sweep_result(logger, row.id, ..., 1, {{"", "riding, no fault", clean.failure}})
end_pose_variant()'srun_row(row, SimScript{}, {{c, tile}}, clean.scripted)has the same shape, though the linter does not flag it. - line 443,
-
""sentinel converted tostd::optionalin three places.differs(),line_with_battle()andTest_AutoStoryFaultSweep::sweep()each writex.empty() ? std::nullopt : std::optional<std::string>(x).RowRun::failureandrow_step_on_sim()'s return are the sentinels; make themstd::optional<std::string>, or at least use one helper. -
fa68899 is in the wrong branch.
add_tests_NavigationSimulation()fixes a function-length failure that #513's threeadd_tests_*lines cause, but it lives only on this FRLG branch. Any other stack that mergesnav/leg-tracehits the same limit. It belongs onnav/leg-trace(#464).make_UNIT_TESTS_ALL()is still at exactly the 50-line limit, so the next suite added fails again. -
Merge order and description. The diff against
frlg/stage20-leaguecarries #513's 1,170 runner lines through thenav/leg-tracemerge, and #464 is still open. Merging this first puts the runner into the FRLG stage branch ahead of its parent. The description is also out of date: #458 is closed, and "the only own-commit file" is no longer true since fa68899.
Nits
- Old name left in two places: the comment at line 735 ("is KNOWN_FAILING's, by name") and the message at line 869 ("KNOWN_FAILING names no such test:"). The list is
known_failing()now. KnownFailingNamesInput,KnownFailingNamesExpectationand the one-rowKNOWN_FAILING_NAMES_CASEStable are empty ceremony: the test takes no input and has one case. A plainUnitTestwith a fixed name says the same. The test itself is not vacuous; it catches a list entry that names no test.whole_row_variant()andend_pose_variant()return lambdas that holdrowandcleanby reference. Correct today; one line in their comments saying the variant must not outlive both would keep it so.
Out of scope, for a follow-up
PokemonFRLG_AutoStoryLavender_Tests.cpp:477still has its ownGAME_ODDS_SEEDS = 30and a hand-written seed loop with its own summary (test_rock_tunnel_game_odds). It is a row test, not a fault sweep, so it is outside this PR's stated change, but it is the remaining copy of whatrun_seeds()does.- Code health reports
test_only_apiadvisories on the shared headers (KnownFailing,SeedRun,SeedsResult,run_seeds,SweepVariant,run_sweep,sweep_clean_first,variant_failure,DEFAULT_SWEEP_SEEDS,SeedStats). These are #513's files, and the runner is test infrastructure used only from test files by design; they need an answer before an upstream handoff, not here.
Depends on #458 (base
frlg/stage20-league) and #464 (nav/leg-trace, merged in for #513's shared runner); the diff also shows #464's commits until it lands in this stack.FRLG's fault sweep carried its own copy of the sweep runner:
SweepFailure/sweep_result(), the clean-first rule, the KNOWN_FAILING wrapper and the 30-seed wild-battle loop. This moves it onto the shared runner from #513: each class now buildsSweepVariants forPokemon::run_sweep(), wild battles run onrun_seeds()(still 30 seeds, now logging mean and worst per measure), the class switch runs undersweep_clean_first(), and the list is aPokemon::KnownFailing. The file keeps only FRLG's rows and how each fault class runs on its sim. Test names and rows are unchanged; the known-failing list keeps its 8 entries, and a new test checks that each entry still names a fault sweep test (KnownFailing::stale()). The only own-commit file isPokemonFRLG_AutoStoryFaultSweep_Tests.cpp; checked by a syntax-only compile with the lab's flags, Fork Draft CI runs the suites.