Skip to content

Start each side's cliff calculation when a report asks for cliffs - #1099

Draft
anth-volk wants to merge 14 commits into
v2plan/c1-38-run-cliff-resultsfrom
v2plan/c1-39-include-cliffs
Draft

anth-volk wants to merge 14 commits into
v2plan/c1-38-run-cliff-resultsfrom
v2plan/c1-39-include-cliffs

Conversation

@anth-volk

Copy link
Copy Markdown
Contributor

Fixes #1098

POST /reports (economy) and POST /reports/{report_id}/runs accept include_cliffs: true again. A request with it starts a cliff calculation for each side's simulation run that has none, and include_cliffs is never stored or keyed.

Stacked on #1096 (C1 · 38: the v2 worker calculates a simulation run's cliff result), and based on its branch. Retarget to main once #1096 merges.

Deploy order:

  1. The Public API's revision bc334f9877b7 (cliff_impacts, Store one cliff result per simulation run in the v2 database, with grants policyengine-api#4327, C1 · 37) must be applied in each environment first.
  2. Then a sim release whose v2 app has Calculate each simulation run's cliff gap and share as its own cliff result #1096 must be the default worker.
  3. Then this.

What a request asking for cliffs does

A simulation run has one cliff result, its cliff gap and cliff share, in a cliff_impacts row that every report naming the run reads (C1 · 24's answer 4b).

  1. Rows. The create or re-run ends on its report run as before, created or joined. In the same transaction, it finds or inserts the cliff calculation of each simulation run that report run names (find_or_insert_report_run_cliffs):

    • INSERT … ON CONFLICT (simulation_run_id) WHERE status <> 'failed' DO NOTHING against uq_cliff_impacts_unfailed, in simulation run ID order;
    • a calculation that's pending, running or succeeded is joined;
    • a failed one never counts, so a new row is inserted;
    • one run on both sides gets one row.

    Two identical requests at once end on one calculation per run. A request the database refuses wrote neither the report run nor its calculations, so a 503 database_unavailable still means nothing was written.

  2. Calls. Only for a calculation it inserted, and before the report run's coordinator, the service spawns the country's simulation function in its cliff mode (Calculate each simulation run's cliff gap and share as its own cliff result #1096). The payload is the run's own SimulationExecutionInput, built as a standalone run's; the context names the calculation (cliff_result_id) under the run's artifact prefix. The service then records the call's ID while the row is pending with none. A calculation runs beside its simulation run, whatever that run's state, and recalculates the simulation from its dataset and policy.

  3. Refused calls. When Modal refuses a cliff call, or doesn't confirm it in 10 s, the service fails that row while it's still pending with no call. The answer is still 201, with that side failed, and the next request asking for cliffs starts another. Because cliff calls start first, a coordinator that Modal refuses (503 modal_call_refused) still leaves them started.

  4. Answer. EconomyReportCreated gains cliffs: each side's cliff_result_id and status. The field is left out when the request didn't ask.

include_cliffs is never stored or keyed

  • EconomyReportRequest.options(), BudgetWindowReportRequest.options() and simulation_options no longer carry it. The coordinator's payload options hold only the SPM selection, and the coordinator never plans cliff columns.
  • The keys and reports.inputs are unchanged by it, which the tests check.
  • A report stored before keys whose inputs hold include_cliffs: true used to answer 422 on re-run. Its re-run now asks for its runs' cliffs. Its stored option is only read, and rerun_scope no longer passes it to the coordinator.

Refusals

  • The interim 422 cliffs_not_available on POST /reports, and on a re-run of such a stored report, is gone (C1 · 24's interim answers A and B).
  • POST /reports/{report_id}/runs takes include_cliffs. A budget window's or household report's re-run asking for it is refused with 422 cliffs_not_available at ["body", "include_cliffs"], since neither calculates cliffs. A budget window's create already refuses the field.

Tests

Red-green throughout: each "(failing)" commit precedes its change. Only the tests this PR adds or changes were run, each by node ID, not whole files or suites. The Postgres tests ran as the simulation service's role, against a database built from #4327's branch (a4be63110). In CI they need public-api-commit to name #4327's merge, which #1096 moves to.

  • Persistence (test_cliff_calculations.py, 14 tests):
    • find or insert per run; a failed row never joined; a succeeded row is the result;
    • two transactions at once insert one row, and two concurrent creates asking for cliffs insert one per run;
    • creates and re-runs (joined, created, forced) write their runs' calculations with the report run;
    • a refused calculation writes no report;
    • a call's ID recorded, or the row failed, only while it's pending with none.
  • Entry (49 tests):
    • creates and re-runs, created or joined, start a call only for an inserted calculation, and none for one being calculated or calculated;
    • one run on both sides gets one calculation, and two identical requests start one per run;
    • keys and stored inputs don't change with the flag, and no coordinator payload carries it;
    • a refused cliff call, an unrecordable call ID, and a failure write that fails;
    • the old 422s are gone, and window and household re-runs are refused;
    • the route and OpenAPI shapes.

Sim's checks (pinned Ruff, rules, quality guards, lock checks, Pyright for the entry and persistence packages) pass.

Review

Two read-only review rounds:

  1. Round 1 found that the cliff rows were written in a transaction of their own, after the report's commit and the coordinator's start. A failure there answered 503 "nothing written" although the report run existed, and forced retries added extra runs. Fixed test-first by writing them in the report run's transaction and starting them first.
  2. Round 2 found no bugs. Its quality findings are fixed: calculations name their run, there is one cliff step for creates and re-runs, and there are tests of concurrent creates and of a refused coordinator on re-run.

Deployment and environment variables

No workflow, Dockerfile, Terraform, v2_app.py or staging check script changes, and the code reads no new environment variable or secret. The service's privileges on cliff_impacts, which /ready checks, come from #1096's access.py.

🤖 Generated with Claude Code

anth-volk and others added 14 commits November 17, 2026 17:32
Cliffs become each simulation run's own calculation (C1 · 24's answer
4b), so the coordinator's and simulation functions' options must hold
only the SPM selection: for a report asking for cliffs, for a re-run of
a report stored with include_cliffs, and for a budget window's options.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
simulation_options takes the SPM selection alone, so neither a report
asking for cliffs nor a re-run of a report stored with include_cliffs
puts it in the coordinator's payload, and a budget window's options no
longer hold include_cliffs: false. Each simulation run's cliffs become a
calculation of their own (C1 · 24's answer 4b).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A request asking for cliffs gets one cliff_impacts row per simulation
run with none that hasn't failed: inserted pending once, then joined;
a failed row is never joined; a succeeded row is the run's result; two
transactions at once insert one row. The store records a row's Modal
call while it is pending with none, and fails it while unstarted.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
find_or_insert_cliff_calculations inserts a pending cliff_impacts row
for each run with none that hasn't failed, against the partial unique
index uq_cliff_impacts_unfailed, and otherwise finds the run's row, in
simulation run ID order. The economy store runs it in a transaction of
its own, and records a row's Modal call, or fails the row, while it is
pending with none, as it does for runs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A report created or re-run with include_cliffs starts a cliff
calculation, through the country's simulation function with the context
naming the cliff result, for each side's simulation run with none, and
joins one being calculated or calculated; two identical requests start
one; include_cliffs changes no key and is never stored; a refused cliff
call fails that calculation and the report is still answered. The 422s
of interim answers A and B give way: a re-run may ask for cliffs, and a
stored report's include_cliffs: true asks for its new runs' cliffs. A
budget window's or household report's re-run asking for cliffs is
refused. The answer names each side's cliff calculation and its status.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
POST /reports with include_cliffs, and a re-run with the re-run body's
new include_cliffs, end on a report run as before; then, in a
transaction of its own, the service finds or creates the cliff
calculation of each simulation run that report run names, and starts
only those it created: the country's simulation function, given the
run's own input and a context naming the cliff calculation (C1 · 38's
cliff mode). A calculation being calculated or calculated is joined, so
two identical requests start one per run. A refused cliff call fails
that calculation and the report is still answered 201. The answer names
each side's cliff calculation and its status, and leaves the field out
when the request didn't ask.

include_cliffs is never stored, in no key and in no options object.
The 422s of C1 · 24's interim answers A and B go: a create may ask for
cliffs, and a report stored before C1 · 27 asking for them is re-run
with its runs' cliffs, its stored option never reaching the
coordinator. A budget window's or household report's re-run asking for
cliffs is refused with 422 cliffs_not_available, as neither calculates
them.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The Simulation Entrypoint's README describes what a create or re-run
asking for cliffs does and answers, and drops the interim 422s; the
persistence README names the cliff calculation lookup and its writes;
the Stage 12 foundation document says the service now starts each
run's cliff calculation. Adds the changelog fragment.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Review round 1 found that writing the cliff calculations in a
transaction of their own, after the report's commit and its
coordinator's start, answers a failure 503 "nothing written" though the
report run exists, and a forced retry adds another forced run each
time. A create or re-run asking for cliffs must insert its runs'
calculations in its own transaction, so a refused calculation writes
nothing and starts no call, and a coordinator Modal refuses still
leaves its runs' calculations started. Also covers a cliff call whose
ID can't be recorded, and a refused one whose failure can't be written.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Review round 1: the cliff calculations were found or created in a
transaction of their own, after the report's commit and its
coordinator's start, so a database failure there answered 503 "nothing
written" though the report run existed, and each forced retry added
another forced run and coordinator. A create or re-run asking for
cliffs now proposes its sides' calculation IDs (ProposedCliffs) and
finds or inserts them in its own transaction, on the two simulation
runs of the report run it ended on (find_or_insert_side_cliffs), so a
refused calculation writes nothing. The service starts the new
calculations before the coordinator, so Modal refusing the coordinator
still leaves them started. The store's separate create goes.

Also from the round: the report kind refused cliffs is typed as
RecordKind, and CliffCalculationState says when a started calculation
is answered failed or pending.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
From review round 2: a found or inserted cliff calculation names the
simulation run it calculates, so the service matches it to its side
without being told the runs again. Also covers two identical creates
asking for cliffs at once (the second loses the report run and joins
its calculations), and a re-run whose coordinator Modal refuses still
starting its runs' cliff calculations.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
From review round 2's code-quality findings:
- FoundCliffCalculation names the simulation run it calculates, so the
  cliff starter takes the calculations found with the report run and
  each side's policy, and no longer matches separately given runs.
- find_or_insert_report_run_cliffs finds or inserts the cliff
  calculations of the runs a report run names, for a create and a
  re-run alike, inside references_checked, with the one check that an
  economy report run names a reform run.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant