Repository navigation
Add country situation preprocessing - #600
Conversation
…uction-hook # Conflicts: # tests/core/test_simulation_builder.py
A Reform inherited TaxBenefitSystem's no-op preprocess_situation, so reformed systems, including the YAML runner's inline-parameter and reforms: cases, skipped the country's hook. Reform now delegates to its baseline. Flat files that declare a group only through its ID column now record it as supplied, matching the loader. IndividualSim's placeholder and default groups are no longer recorded as supplied. A hook that returns something other than a dict raises a TypeError naming the hook. The docs describe dataset provenance, axes and default_period accurately. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
PolicyEngine/policyengine-core#600 now includes Core's main (3.34.0) and delegates the hook through reforms, so main's policyengine-us imports against it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Remove Simulation.input_group_entities and its recording in the builder, build_from_dataset and IndividualSim. The country model now builds marital units that agree with its tax roles, so no formula needs to know which groups the caller supplied. Datasets still bypass the hook. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
PavelMakarchuk
left a comment
There was a problem hiding this comment.
Verdict: REQUEST_CHANGES — the hook is well placed and backward compatible, but a Reform runs the hook against the unreformed baseline, and two claimed behaviors have no test.
Should address
-
A
Reformruns the hook on the baseline, so the hook cannot see the reform (policyengine_core/reforms/reform.py:150-153).self.baseline.preprocess_situation(...)binds the hook to the baseline system. Any parameter or variable that the hook reads is the pre-reform one. I checked this with a hook that reads a parameter: under a reform that sets the parameter to 0.99, the hook still read the baseline 0.15, both forReform(baseline)and for the system that_get_tax_benefit_systembuilds for the YAML runner.
This matters for the consumer. The hook in PolicyEngine/policyengine-us#10000 readssystem.parameters(...).gov.irs.dependent.ineligible_ageandsystem.get_variable(...).formulas. A YAML test with an inline parameter or areforms:entry would build marital units from baseline values, while a system that applies a reform in place (TaxBenefitSystem(reform=...)) gives the hook the reformed values. The two paths then disagree.
Options: call the baseline class's hook with the reform asself(walkbaselineto the first non-Reformsystem, thentype(root).preprocess_situation(self, situation, default_period)), or state in the docstring and docs that the hook must not read parameters or variables. Add a test for the chosen behavior. -
No test that a dataset skips the hook. The PR body, the docstring (
policyengine_core/taxbenefitsystems/tax_benefit_system.py:153-164) anddocs/python_api/simulations.md:27-35all claim it. The only "skip" assertion (tests/core/test_simulation_builder.py:163-166) callsbuild_default_simulation, which never touches a situation, so it cannot fail. Please add a test that buildsSimulation(dataset=...)with a recording hook and asserts zero calls, and replace thebuild_default_simulationassertion with a variable-onlybuild_from_dict(for example{"salary": {"2024-01": 100}}). -
No test that the hook receives
axes. The docs and the docstring state "with anyaxesstill present". A hook that fills groups must handle this case, so please pin it with one assertion.
Suggestions
- Tolerant lookup for stand-in systems (
policyengine_core/simulations/simulation_builder.py:229). The direct attribute access raisesAttributeError: 'System' object has no attribute 'preprocess_situation'for an object that only duck-types aTaxBenefitSystem. Every realTaxBenefitSystemsubclass and everyReforminherits the default, so country packages are safe. The one known stand-in is the test fixture that PolicyEngine/policyengine-us#10850 fixes. Agetattr(tax_benefit_system, "preprocess_situation", None)guard would remove the release-order dependency, but I do not think Core must support systems outside its class hierarchy. Optional. - Docstring precision. "With singular entity aliases expanded" holds only through
build_from_dict. A directbuild_from_entitiescall gives the hook the raw keys (household, nothouseholds). The hook also runs before the unknown-entity and missing-person checks, so it can receive a situation with no person key. One sentence on each would help hook authors. - PR body. "Not run locally for this revision" — see the results below; the body can now state them.
What I verified
- Call site. One call, in
SimulationBuilder.build_from_entities(simulation_builder.py:229-236), aftercopy.deepcopy(input_dict)(line 213) and beforeaxesis popped and entities are validated. - Paths, measured with a recording hook on the country template:
build_from_dictwith entity keys: 1 call.Simulation(situation=...): 1 call, caller's dict unchanged after a nested mutation in the hook.- Situation with
axes: 1 call,axespresent in the hook input, 3 persons built. - YAML runner (
tools/test_runner.py:261) goes throughbuild_from_dict. clone(),get_branch()andcalculate()after construction: 0 more calls.- Variable-only
build_from_dict,build_default_simulation, andSimulation(dataset=...)(flat-file dataset, 2 households): 0 calls. Reform(system)and a nestedReform(Reform(system)): 1 call each, delegated.- A non-dict return raises the
TypeError; a hook that drops the person key falls through to the usualSituationParsingError.
- Backward compatibility. The default hook returns its input, so a country package or an older policyengine-us that does not override it builds the same simulation. The country template without an override builds as before.
- Tests.
tests/core/test_simulation_builder.py: 49 passed. All tests intests/corethat match builder, reform, runner, simulation, axes or individual: 580 passed. - Repo rules.
changelog.d/situation-preprocessing.added.mdmatches the towncrier setup inpyproject.toml(addedgives a minor bump).docs/python_api/simulations.mddocuments the hook. - CI at 1:57pm EST on Nov 12: 18 checks pass, 1 pending (Lint), on head
f37aef590a. - Merge order. I confirm that PolicyEngine/policyengine-us#10850 must merge before a Core release with this change: with this head installed,
policyengine_us/tests/core/test_social_security_input_state_machine.pyfails on policyengine-usmainwith theAttributeErrorabove, and passes with that fix.
A Reform now calls its country system's hook with the reform as the system, so the hook reads the reformed parameters and variables; a hook set on the country system instance itself stays bound to it. Document where singular aliases are expanded and that the hook runs before entity validation. Test datasets and variable-only builds skipping the hook, the hook receiving axes, nested reforms and a reformed parameter. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Thanks, Pavel. Changes for each point:
Suggestions:
|
PavelMakarchuk
left a comment
There was a problem hiding this comment.
Verdict: APPROVE — all three round-1 items are fixed and tested; only optional suggestions remain.
I re-reviewed head 4aaae3af96 (one new commit since f37aef590a).
Round-1 items
| # | Item | Status |
|---|---|---|
| 1 | A Reform ran the hook on the baseline |
Resolved. The hook now runs with the reform as the system (policyengine_core/reforms/reform.py:150-164), with a test (tests/core/test_simulation_builder.py:276). |
| 2 | No test that a dataset skips the hook | Resolved (tests/core/test_simulation_builder.py:194); the build_default_simulation assertion is now a variable-only build_from_dict (:167). |
| 3 | No test that the hook receives axes |
Resolved (tests/core/test_simulation_builder.py:171). |
| S1 | Tolerant lookup for stand-in systems | Declined with a reason I accept: PolicyEngine/policyengine-us#10850 is merged, and every TaxBenefitSystem and Reform inherits the default. |
| S2 | Docstring precision (aliases, no-persons case) | Resolved in the docstring and docs/python_api/simulations.md. |
| S3 | PR body test statement | Resolved. |
Suggestions
- A reform class that overrides the hook is skipped when another reform wraps it (
policyengine_core/reforms/reform.py:156-158). The loop walks past everyReformto the country system, so forOuter(HookReform(system))the hook ofHookReformdoes not run; the country system's hook runs instead. Before this commit the chain reached it. I know of no reform that overrides the hook, so this is optional: stop the walk at the first system whose class defines its ownpreprocess_situation, or add one sentence to the docstring that only the country system's hook is used. - An instance-level hook does not see the reform. A hook set on the country system instance stays bound to that instance (
reform.py:159-161), so it reads baseline parameters under a reform. The code comment says so; one clause in the docstring would tell hook authors too. test_reform_delegates_preprocessing_to_its_baseline(tests/core/test_simulation_builder.py:243) now tests "runs the country hook", not "delegates to the baseline". A rename would match the new behavior.
What I verified
- Item 1, measured with a hook that reads
taxes.income_tax_rate(baseline 0.15, reform 0.99):- baseline system: 0.15;
Reform(baseline): 0.99; - nested reforms in both orders (0.50 over 0.99, 0.99 over 0.50): the outer reform's value; a no-op reform over a rate reform: 0.99;
- the system that
_get_tax_benefit_systembuilds for the YAML runner, with one and with two reforms: 0.99; TaxBenefitSystem(reform=...)(class reform and dict reform, applied in place): 0.99;Simulation(situation=...)on a reform: 0.99;- the baseline again after all of the above: still 0.15, so no leak into the baseline.
All paths now agree. A hook that reads a variable sees a reform'sneutralize_variableon the same paths.
- baseline system: 0.15;
- Call count. Each build calls the hook exactly once, including nested reforms and a situation with
axes(3 persons built,axespresent in the hook input).Simulation.clone(),get_branch()andcalculate()add 0 calls.Reform.clone()andTaxBenefitSystem.clone()keep the hook and their own parameter values. - Default path. The country template without an override builds the same simulation, alone and under one or two reforms. A non-dict return still raises the
TypeError, on a system and on a reform. The caller's dict is unchanged in every case. - Instance hook. A hook set on the system instance is called once through the system, a reform and a nested reform.
- Consumer. With this head installed, PolicyEngine/policyengine-us#10000 at
e7d1a89097builds the same marital units under a reform throughSimulation(reform=...), aReformsystem, a nested reform and a YAML test with an inline parameter (dependent age limit raised to 30: a parent 45 and child 25 are no longer paired in all four). - Tests.
tests/core/test_simulation_builder.py: 52 passed. All tests intests/corethat match builder, reform, runner, simulation, axes or individual: 583 passed.ruff format --checkandruff checkpass. - CI. All 19 checks pass on
4aaae3af96(run finished 4:08pm EST on Nov 12). - Mergeability.
git merge-tree --write-tree origin/master HEADis clean against7c84d941. - Repo rules.
changelog.d/situation-preprocessing.added.mdis present and unchanged.
Only the country system's hook runs: a Reform subclass that overrides it is skipped when another reform wraps it, and a hook set on the country system instance reads the baseline's values. Rename the reform test to match what it checks. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Thanks, Pavel. I took the two docstring suggestions and the rename:
|
…ax-role fallback (#10000) * Preserve marital inputs and infer missing units from explicit pairs * Pin policyengine-core to the updated situation-hook PR head PolicyEngine/policyengine-core#600 now includes Core's main (3.34.0) and delegates the hook through reforms, so main's policyengine-us imports against it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Pair married families and keep inferred units through snapshots When marital units are omitted, pair the two adults of a tax unit whose family has an explicit is_married input, so role-less couples marked married keep SSI couple treatment (DC OSSP Cases 2 and 5). Source-input snapshots now record which groups the caller supplied, so a reloaded simulation keeps inferred marital units as inferred and its tax spouse. Count NumPy and numeric true inputs, pair role inputs with overlapping periods, and leave malformed input to Core's validation. Move the new SSI cases to the end of their file, document the rules after the page introduction, and regenerate uv.lock with uv 0.13.4. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Pair the tax-role couple when a situation omits marital units When nothing marks a couple, the marital-unit hook now pairs the head and spouse that is_tax_unit_head and is_tax_unit_spouse infer from age order, reading the same inputs and dependent age limits, after explicit role inputs, a complete tax_unit_role_input and a married family. Couples sent without marriage information keep SSI and Medicaid couple treatment, and a parent's adult child who meets the qualifying child age test stays out. Drop the supplied-membership provenance (policyengine-core PR 600 no longer records it), the snapshot provenance fields, the spouse gate that #10555 replaced and the git pin on Core, which becomes a released minimum once Core ships the hook. Replace the SSI cases with helper tests and two end-to-end YAML cases. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Describe omitted marital units as built by the role rules The tax-role comments and docs said a situation that omits marital units puts everyone in one unit. The hook now builds those units by the same rules as the role fallback, so say so. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Address review of the marital-unit hook Check only the years the inputs are dated for, so a payload dated for a year other than Simulation's default input year keeps its couple wherever it is listed, and read a person's first input for earlier years. Give everyone in a tax unit and household their own unit, instead of raising, when inputs can't settle one static couple. Respect false head and spouse inputs in the role fallback. Use the reform's dependent age limits. Add tests for dated payloads, two couples, integer ids, false role inputs and the adult age, list the grandchild first in the SSI couple case, add a parent and disabled adult child who aren't a joint claim, and give the four people in the home equity Case 21 separate marital units. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Let the second California couple qualify on their own income The case expected the second couple to fail because the income reader summed one marital unit holding all five people. With marital units built for situations that omit them, the couple share their own unit, have no income and qualify. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Read the defaults Core fills in when anyone supplies an input Once anyone in a situation supplies a role, student or age input for a year, Core gives everyone else that variable's default instead of running its formula. Step 4 now reads the same defaults, so a false role filled that way stops the pairing and the units agree with the roles. Add tests for a grandchild's false role input, another person's student input and a reform to the dependent age limit, and list the lone spouse input on the oldest adult as a known limitation. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Require the Core release with the situation hook Raise the policyengine-core floor to 3.35.0, the first release with TaxBenefitSystem.preprocess_situation, and regenerate uv.lock with CI's uv 0.13.4. Without the floor, an older Core would skip the hook and keep one marital unit. Document that marital units follow the years the inputs are dated for, so a calculation for another year can give roles the units don't match, and pin the undated case with a test. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Pair couples from filing status and fix tests that relied on one marital unit The first CI run with the hook active showed that it split couples whose filing status says they are married, and that six existing tests depended on Core putting everyone in one marital unit. - A JOINT filing_status input skips step 4's qualifying child age test, since only a married couple files jointly (26 USC 6013(a)). - The two separate filers of a household are paired, since only married people file separately (26 USC 1(d)); filers in different households stay apart because a marital unit is a married, co-habiting couple. - Step 4 reads an is_child input, as head_or_spouse_candidates does. - HUD case sends its couple's marital unit; the role-inference reference applies the age test; the Arkansas property test puts its two filers in one household; three Montana elderly credit cases count the individual SSI that the single default unit hid. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Read a changing filing status as no status, and test each separate-filer rule A filing_status input that changed between periods raised the ambiguity exception inside the same try as steps 1 to 3, so it cancelled explicit head and spouse inputs too: an aged couple with {2025: JOINT, 2026: SEPARATE} lost their joint SSI claim. A changing status now counts as no status, leaving the steps that don't read it to settle the couple. Each separate-filer rule now has a test that fails when the rule is removed: a separated filer, a family with is_married false, a return with no clear filer, a head input naming the filer, an input dependent or child not being the filer, and a filer already in a couple. The docs say who the filer is and add two limitations: the separate-filer pairing reads no age or relationship, and SINGLE, HEAD_OF_HOUSEHOLD and SURVIVING_SPOUSE inputs aren't read as unmarried. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Pin how a changing filing status is read, and document two edge cases Two tests now fix the documented rule that a filing_status changing between periods counts as no status: a joint return dated {2024: SINGLE, 2025: JOINT} with a 20-year-old student spouse pairs no one, and two separate filers, one of them {2024: SEPARATE, 2025: SINGLE}, pair no one. Reading a changing status as JOINT in any year, as SEPARATE in any year, or as its last value each fails a test. The docs now say that a dependent also marked head or spouse gives own units only beside a head and spouse pair (a lone spouse input on a dependent pairs through step 4, as the roles do), and that a filing status dated for more than one year must be constant to count. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Apply the age test in the shared age-order reference test_tax_unit_role_input_properties.py's _age_rule, which main's new test_tax_unit_spouse_separation_invariants.py also uses, made the next-oldest adult the spouse with no qualifying child age test, as Core's single default marital unit allowed. With marital units built for situations that omit them, an adult under 19 and younger than the head meets the age test (26 USC 152(c)(3)(A)(i)) and isn't the spouse. The reference now skips that candidate; the separation rule is unchanged. Both docstrings no longer say Core puts everyone in one marital unit. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Test a separated head, and document a lone spouse input beside a separated adult From Pavel's approval: no hook test failed when step 4 ignored a separated head, so one row now pins it. A lone true is_tax_unit_spouse input returns the pair before step 4 checks separation, so the oldest other adult is paired even when separated; these inputs contradict each other, and the limitation is now documented rather than changing the approved code. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Summary
Country models can't fill in entity groups a situation leaves out the same way across the Household API,
Simulation(situation=...)and the YAML runner. Core builds whatever the situation gives it and puts everyone into one group for each omitted group entity. For policyengine-us that makes a whole household one marital unit, so a parent and an adult child are treated as a married couple.This PR adds an opt-in
TaxBenefitSystem.preprocess_situation(situation, default_period)hook.SimulationBuilder.build_from_entitiescalls it before building populations:axesstill present, and the builder's default period as set (None, a string, an integer or aPeriod). Throughbuild_from_dictandSimulation(situation=...), singular entity aliases are already expanded; a directbuild_from_entitiescall passes the caller's keys as given. It runs before the builder validates entities.TypeErrornaming the system.Reformruns its country system's hook with the reform as the system, so the hook reads the reformed parameters and variables.ReforminheritsTaxBenefitSystem's no-op, so without this a reformed system, and the YAML runner's inline-parameter andreforms:cases, would skip the country's hook. A hook set on the country system instance itself stays bound to that instance.Prerequisite for PolicyEngine/policyengine-us#9999; the consumer is PolicyEngine/policyengine-us#10000, which needs a Core release that includes this hook.
Changes since the first revision
The first revision also recorded
Simulation.input_group_entities, the group entities a caller or dataset supplied, so formulas could tell supplied memberships from defaults. policyengine-us#10000 now builds marital units that agree with its tax roles, so no formula reads that record, and this revision removes it with its builder,build_from_datasetandIndividualSimcode, tests and documentation. Core's master branch is merged in.After review, a
Reformruns the country's hook with the reformed system (it ran on the baseline before), the docstring and docs say where aliases are expanded and that the hook runs before entity validation, and new tests cover datasets, variable-only builds,axesand the reformed parameter.Before release
policyengine-us's Social Security state-machine test backend copies
build_from_entitiesinto a stand-in system. PolicyEngine/policyengine-us#10850, now merged, gives that system the hook.Tests
tests/core/test_simulation_builder.py:axes;Simulation(situation=...)runs it, while a variable-onlybuild_from_dictandSimulation(dataset=...)don't;tests/core/test_simulation_builder.pypasses locally (52 tests).ruff formatandruff checkpass on the changed files.🤖 Generated with Claude Code