Repository navigation
Conversation
Cliffs become each simulation run's own calculation (C1 · 24's answer 4b), so the coordinator's and simulation functions' options must hold only the SPM selection: for a report asking for cliffs, for a re-run of a report stored with include_cliffs, and for a budget window's options. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
simulation_options takes the SPM selection alone, so neither a report asking for cliffs nor a re-run of a report stored with include_cliffs puts it in the coordinator's payload, and a budget window's options no longer hold include_cliffs: false. Each simulation run's cliffs become a calculation of their own (C1 · 24's answer 4b). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A request asking for cliffs gets one cliff_impacts row per simulation run with none that hasn't failed: inserted pending once, then joined; a failed row is never joined; a succeeded row is the run's result; two transactions at once insert one row. The store records a row's Modal call while it is pending with none, and fails it while unstarted. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
find_or_insert_cliff_calculations inserts a pending cliff_impacts row for each run with none that hasn't failed, against the partial unique index uq_cliff_impacts_unfailed, and otherwise finds the run's row, in simulation run ID order. The economy store runs it in a transaction of its own, and records a row's Modal call, or fails the row, while it is pending with none, as it does for runs. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A report created or re-run with include_cliffs starts a cliff calculation, through the country's simulation function with the context naming the cliff result, for each side's simulation run with none, and joins one being calculated or calculated; two identical requests start one; include_cliffs changes no key and is never stored; a refused cliff call fails that calculation and the report is still answered. The 422s of interim answers A and B give way: a re-run may ask for cliffs, and a stored report's include_cliffs: true asks for its new runs' cliffs. A budget window's or household report's re-run asking for cliffs is refused. The answer names each side's cliff calculation and its status. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
POST /reports with include_cliffs, and a re-run with the re-run body's new include_cliffs, end on a report run as before; then, in a transaction of its own, the service finds or creates the cliff calculation of each simulation run that report run names, and starts only those it created: the country's simulation function, given the run's own input and a context naming the cliff calculation (C1 · 38's cliff mode). A calculation being calculated or calculated is joined, so two identical requests start one per run. A refused cliff call fails that calculation and the report is still answered 201. The answer names each side's cliff calculation and its status, and leaves the field out when the request didn't ask. include_cliffs is never stored, in no key and in no options object. The 422s of C1 · 24's interim answers A and B go: a create may ask for cliffs, and a report stored before C1 · 27 asking for them is re-run with its runs' cliffs, its stored option never reaching the coordinator. A budget window's or household report's re-run asking for cliffs is refused with 422 cliffs_not_available, as neither calculates them. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The Simulation Entrypoint's README describes what a create or re-run asking for cliffs does and answers, and drops the interim 422s; the persistence README names the cliff calculation lookup and its writes; the Stage 12 foundation document says the service now starts each run's cliff calculation. Adds the changelog fragment. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Review round 1 found that writing the cliff calculations in a transaction of their own, after the report's commit and its coordinator's start, answers a failure 503 "nothing written" though the report run exists, and a forced retry adds another forced run each time. A create or re-run asking for cliffs must insert its runs' calculations in its own transaction, so a refused calculation writes nothing and starts no call, and a coordinator Modal refuses still leaves its runs' calculations started. Also covers a cliff call whose ID can't be recorded, and a refused one whose failure can't be written. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Review round 1: the cliff calculations were found or created in a transaction of their own, after the report's commit and its coordinator's start, so a database failure there answered 503 "nothing written" though the report run existed, and each forced retry added another forced run and coordinator. A create or re-run asking for cliffs now proposes its sides' calculation IDs (ProposedCliffs) and finds or inserts them in its own transaction, on the two simulation runs of the report run it ended on (find_or_insert_side_cliffs), so a refused calculation writes nothing. The service starts the new calculations before the coordinator, so Modal refusing the coordinator still leaves them started. The store's separate create goes. Also from the round: the report kind refused cliffs is typed as RecordKind, and CliffCalculationState says when a started calculation is answered failed or pending. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
From review round 2: a found or inserted cliff calculation names the simulation run it calculates, so the service matches it to its side without being told the runs again. Also covers two identical creates asking for cliffs at once (the second loses the report run and joins its calculations), and a re-run whose coordinator Modal refuses still starting its runs' cliff calculations. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
From review round 2's code-quality findings: - FoundCliffCalculation names the simulation run it calculates, so the cliff starter takes the calculations found with the report run and each side's policy, and no longer matches separately given runs. - find_or_insert_report_run_cliffs finds or inserts the cliff calculations of the runs a report run names, for a create and a re-run alike, inside references_checked, with the one check that an economy report run names a reform run. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #1098
POST /reports(economy) andPOST /reports/{report_id}/runsacceptinclude_cliffs: trueagain. A request with it starts a cliff calculation for each side's simulation run that has none, andinclude_cliffsis never stored or keyed.Stacked on #1096 (C1 · 38: the v2 worker calculates a simulation run's cliff result), and based on its branch. Retarget to
mainonce #1096 merges.Deploy order:
bc334f9877b7(cliff_impacts, Store one cliff result per simulation run in the v2 database, with grants policyengine-api#4327, C1 · 37) must be applied in each environment first.What a request asking for cliffs does
A simulation run has one cliff result, its cliff gap and cliff share, in a
cliff_impactsrow that every report naming the run reads (C1 · 24's answer 4b).Rows. The create or re-run ends on its report run as before, created or joined. In the same transaction, it finds or inserts the cliff calculation of each simulation run that report run names (
find_or_insert_report_run_cliffs):INSERT … ON CONFLICT (simulation_run_id) WHERE status <> 'failed' DO NOTHINGagainstuq_cliff_impacts_unfailed, in simulation run ID order;Two identical requests at once end on one calculation per run. A request the database refuses wrote neither the report run nor its calculations, so a 503
database_unavailablestill means nothing was written.Calls. Only for a calculation it inserted, and before the report run's coordinator, the service spawns the country's simulation function in its cliff mode (Calculate each simulation run's cliff gap and share as its own cliff result #1096). The payload is the run's own
SimulationExecutionInput, built as a standalone run's; the context names the calculation (cliff_result_id) under the run's artifact prefix. The service then records the call's ID while the row is pending with none. A calculation runs beside its simulation run, whatever that run's state, and recalculates the simulation from its dataset and policy.Refused calls. When Modal refuses a cliff call, or doesn't confirm it in 10 s, the service fails that row while it's still pending with no call. The answer is still 201, with that side
failed, and the next request asking for cliffs starts another. Because cliff calls start first, a coordinator that Modal refuses (503modal_call_refused) still leaves them started.Answer.
EconomyReportCreatedgainscliffs: each side'scliff_result_idandstatus. The field is left out when the request didn't ask.include_cliffsis never stored or keyedEconomyReportRequest.options(),BudgetWindowReportRequest.options()andsimulation_optionsno longer carry it. The coordinator's payload options hold only the SPM selection, and the coordinator never plans cliff columns.reports.inputsare unchanged by it, which the tests check.inputsholdinclude_cliffs: trueused to answer 422 on re-run. Its re-run now asks for its runs' cliffs. Its stored option is only read, andrerun_scopeno longer passes it to the coordinator.Refusals
cliffs_not_availableonPOST /reports, and on a re-run of such a stored report, is gone (C1 · 24's interim answers A and B).POST /reports/{report_id}/runstakesinclude_cliffs. A budget window's or household report's re-run asking for it is refused with 422cliffs_not_availableat["body", "include_cliffs"], since neither calculates cliffs. A budget window's create already refuses the field.Tests
Red-green throughout: each "(failing)" commit precedes its change. Only the tests this PR adds or changes were run, each by node ID, not whole files or suites. The Postgres tests ran as the simulation service's role, against a database built from #4327's branch (
a4be63110). In CI they needpublic-api-committo name #4327's merge, which #1096 moves to.test_cliff_calculations.py, 14 tests):inputsdon't change with the flag, and no coordinator payload carries it;Sim's checks (pinned Ruff, rules, quality guards, lock checks, Pyright for the entry and persistence packages) pass.
Review
Two read-only review rounds:
Deployment and environment variables
No workflow, Dockerfile, Terraform,
v2_app.pyor staging check script changes, and the code reads no new environment variable or secret. The service's privileges oncliff_impacts, which/readychecks, come from #1096'saccess.py.🤖 Generated with Claude Code