Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion .github/scripts/test_alembic_v2_lifecycle.sh
Original file line number Diff line number Diff line change
Expand Up @@ -11,4 +11,5 @@ uv run --frozen --extra dev pytest -q \
tests/integration/test_alembic_v2_saved_item_keys.py \
tests/integration/test_alembic_v2_user_types.py \
tests/integration/test_alembic_v2_withheld_area_codes.py \
tests/integration/test_alembic_v2_shared_report_keys.py
tests/integration/test_alembic_v2_shared_report_keys.py \
tests/integration/test_alembic_v2_cliff_impacts.py
1 change: 1 addition & 0 deletions changelog.d/simulation-run-cliff-impacts.added.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
The v2 database can store one cliff result per simulation run, its cliff gap and cliff share, with the state of its calculation. The simulation service may start each calculation, its Modal workers may write the result, and the Public API may read it.
77 changes: 75 additions & 2 deletions docs/engineering/skills/alembic-migrations.md
Original file line number Diff line number Diff line change
Expand Up @@ -255,8 +255,8 @@ and a revision runs once per database, so it can't repair drift.
| `policyengine_v2_runtime` | The Public API on Cloud Run |
| `policyengine_v2_catalog_writer` | `scripts/publish_v2_metadata_catalog.py`, which also runs `ANALYZE` on seven catalog tables after every publication, with `MAINTAIN` (PostgreSQL 17) |
| `policyengine_stage12_<environment>` | The simulation API and its Modal worker, on the temporary comparison-run tables |
| `policyengine_v2_simulation_service` | The simulation API on Cloud Run: creates simulations, reports, their runs, budget-window runs and household jobs, and reads the policy, dynamic, household and catalog rows a run needs. It updates only the status, Modal call ID, error and completion columns of the runs and household jobs whose Modal calls it starts, and the status, error and completion columns of a budget-window run, which it fails at once when Modal refuses the calls of the window's first years |
| `policyengine_v2_modal_worker` | The simulation API's Modal workers: write their runs' and household jobs' status and results, and budget-window runs' status, and read the rows a run needs. It updates only a run's status, Modal call ID, error, time and result columns, and a budget-window run's status, error and time columns: each year's coordinator starts the window's next unstarted year when its own year finishes, the last year to finish marks the window succeeded, and a failed year fails it. The workers' code, not the database, keeps a worker to its own run |
| `policyengine_v2_simulation_service` | The simulation API on Cloud Run: creates simulations, reports, their runs, budget-window runs, household jobs and simulation runs' cliff calculations, and reads the policy, dynamic, household and catalog rows a run needs. It updates only the status, Modal call ID, error and completion columns of the runs, household jobs and cliff calculations whose Modal calls it starts, and the status, error and completion columns of a budget-window run, which it fails at once when Modal refuses the calls of the window's first years |
| `policyengine_v2_modal_worker` | The simulation API's Modal workers: write their runs', household jobs' and cliff calculations' status and results, and budget-window runs' status, and read the rows a run needs. It updates only a run's or cliff calculation's status, Modal call ID, error, time and result columns, and a budget-window run's status, error and time columns: each year's coordinator starts the window's next unstarted year when its own year finishes, the last year to finish marks the window succeeded, and a failed year fails it. The workers' code, not the database, keeps a worker to its own run |

Every v2 table has an entry for every role, so "no access" is a decision, not
an omission. A privilege is declared on the whole table, or with
Expand Down Expand Up @@ -456,6 +456,8 @@ it publishes that version, with nothing to restore it. Downgrading past the
revision is therefore safe only while no version is pruned (no
`tax_benefit_model_versions` row has `catalog_values_pruned_at` set).
From the head, such a downgrade first runs the later revisions' downgrades:
`bc334f9877b7`'s drops `cliff_impacts` with every simulation run's cliff
results (see "API v2 simulation-run cliff impacts"),
`f519a77ba122`'s drops `user_invitations` and needs no recorded change with
source `invitation` (see "API v2 user invitations"),
`94dc43079186`'s drops every user's report-run record, the simulation and
Expand Down Expand Up @@ -1012,3 +1014,74 @@ change's source is `invitation`.
`test_user_invitations_revision_adds_invitations_and_dismissals_only` and
`test_user_invitations_revision_records_invitation_changes_until_downgraded`
in `tests/integration/test_alembic_v2_user_types.py` check both.

### API v2 simulation-run cliff impacts

Revision `bc334f9877b7` (parent `f519a77ba122`) was autogenerated, with its
ID given by `--rev-id`, after the `CliffImpact` model was added. Each
simulation run can have one cliff result, linked to the run and not to any
report, which every report naming the run shares: cliffs need marginal tax
rates, which recalculate each household with extra earnings, so a shared run
never stores the cliff columns (US national runs keep their region-group
split), and a request that asks for cliffs starts a separate calculation of
the run on its dataset, policy and dynamic.

It creates `cliff_impacts`, one row per cliff calculation of a simulation run:

- `simulation_run_id`, a foreign key to `simulation_runs` with `CASCADE`, as
`poverty` and `inequality` have: the result is part of its run, and nothing
references it. A run can be deleted only once no report run names it.
- The calculation's state: `status` (`pending`, `running`, `succeeded` or
`failed`, in the `v2_simulation_status` type simulation runs use),
`modal_call_id`, `error_message` (`code: message`, as runs record theirs),
`started_at` and `completed_at`, with the run tables' check that a
succeeded or failed row has `completed_at`
(`ck_cliff_impacts_finished_completion`).
- The result: `cliff_gap`, the weighted total of the net income people would
lose from a small rise in their earnings, and `cliff_share`, the weighted
number of people on a cliff divided by the weighted number of adults, both
double precision. A succeeded row has both, and no other row has either
(`ck_cliff_impacts_result_when_succeeded`). Neither is range-checked: .py
doesn't keep the share to at most 1, and a filtered simulation with no
adults gives NaN.
- `uq_cliff_impacts_unfailed`, a partial unique index on `simulation_run_id`
where `status <> 'failed'` (also declared for SQLite): a run has at most one
calculation that hasn't failed, and so at most one result. The simulation
service is to start one with `INSERT ... ON CONFLICT (simulation_run_id)
WHERE status <> 'failed' DO NOTHING`, so that two requests at once start
one calculation and the second joins it. A failed calculation never counts:
it stays as a record, and the next request inserts another.
- An index on `(status, started_at)`, as `simulation_runs` has, for finding
unfinished calculations by age, as the simulation API's stuck-run sweep
does for runs: a calculation whose worker ended without recording its
outcome stays unfinished, and keeps later requests from starting another,
until something fails it.

A result has no versions of its own: it inherits its run's catalog and
package versions, so it is calculated only for a run on the versions the
simulation API serves. The simulation service may read and insert rows and update their
status, Modal call ID, error and completion columns; the Modal worker may read
them and update their state and result columns, never insert them; the
runtime role may read them; neither simulation role may change the run a row
names or delete a row. The reconcile step after each schema upgrade grants
them. Until the simulation API writes the table, no run has a cliff
calculation, and its cliffs stay refused. `report_runs`' four cliff columns
stay until a later revision drops them, once the Public API reads cliffs from
here.

It has one post-generation correction, commented in the revision: `status`
reuses the baseline's `v2_simulation_status` type
(`postgresql.ENUM(..., create_type=False)`), as `1356f658347d` does for
`simulation_runs.status`; the generated `create_table` would create the type
again and fail. So the downgrade, which drops only the table, leaves the type
to `simulation_runs`.

The revision writes no data and changes no other table, and the previous
Public API revision, which doesn't map the table, keeps working at it. The
downgrade drops the table with every calculation and result; a later
calculation of the same runs on the same versions gives the same numbers.
`tests/integration/test_alembic_v2_cliff_impacts.py` checks that nothing
else changes, that the downgrade leaves the type and an upgrade works again,
and the index, checks and delete rule;
`tests/integration/test_v2_simulation_role_privileges.py` runs the writes
and reads as the roles that do them.
4 changes: 3 additions & 1 deletion docs/migration/stage-12a-user-login.md
Original file line number Diff line number Diff line change
Expand Up @@ -770,7 +770,9 @@ admin; `scripts/set_v2_user_type.py` makes another admin afterwards.
`users.primary_country`, so it fails while legacy-only rows, any simulation
or report associations, or any unauthenticated user exist. Removing them
first deletes users' saved work. From the head, the later revisions'
downgrades run first: the user invitations revision (`f519a77ba122`) drops
downgrades run first: the cliff impacts revision (`bc334f9877b7`) drops
every simulation run's cliff results, the user invitations revision
(`f519a77ba122`) drops
every invitation and dismissal and needs no change with source
`invitation`, the shared reports revision (`94dc43079186`) drops
every user's report-run record, every simulation and report key, the runs'
Expand Down
6 changes: 4 additions & 2 deletions docs/migration/stage-9-v2-metadata.md
Original file line number Diff line number Diff line change
Expand Up @@ -835,8 +835,10 @@ pruning").

A schema downgrade past revision `334859ef4fec` drops
`catalog_last_published_at` and `catalog_values_pruned_at`. From the head,
it first runs the later revisions' downgrades: `f519a77ba122`'s drops
`user_invitations` and needs no recorded change with source `invitation`,
it first runs the later revisions' downgrades: `bc334f9877b7`'s drops
`cliff_impacts` with every simulation run's cliff results, `f519a77ba122`'s
drops `user_invitations` and needs no recorded change with source
`invitation`,
`94dc43079186`'s drops
`user_report_run_associations` with every user's report-run record, the
simulation and report keys, the runs' `forced` markers, the window runs' .py
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,86 @@
"""record each simulation run's cliff calculations

Revision ID: bc334f9877b7
Revises: f519a77ba122
Create Date: 2026-11-17 17:38:09.806260
Generation: uv run alembic -c alembic-v2.ini revision --autogenerate --rev-id bc334f9877b7

One post-generation correction, commented where it is made:
cliff_impacts.status reuses the existing v2_simulation_status type instead of
creating it again.

Each simulation run can have one cliff result: its cliff gap and cliff share,
linked to the run and not to any report. It adds cliff_impacts, one row per
cliff calculation of a simulation run, holding the calculation's state
(status, Modal call, error and times) and, once it succeeds, the two numbers.
A partial unique index allows one calculation per run that hasn't failed, so
identical requests join it; a failed calculation never counts. Deleting a run
deletes its calculations. A result has no versions of its own: it inherits its
run's.

It writes no data and changes no other table, and the previous Public API
revision, which doesn't map the table, keeps working at it. The downgrade
drops the table with every calculation and result, and leaves the type,
which simulation_runs still uses.
"""

from typing import Sequence, Union

from alembic import op
import sqlalchemy as sa
import sqlmodel
from sqlalchemy.dialects import postgresql


revision: str = 'bc334f9877b7'
down_revision: Union[str, None] = 'f519a77ba122'
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None


def upgrade() -> None:
# ### commands auto generated by Alembic - please adjust! ###
op.create_table('cliff_impacts',
sa.Column('id', sa.Uuid(), nullable=False),
sa.Column('created_at', sa.DateTime(timezone=True), server_default=sa.text('now()'), nullable=False),
sa.Column('updated_at', sa.DateTime(timezone=True), server_default=sa.text('now()'), nullable=False),
sa.Column('simulation_run_id', sa.Uuid(), nullable=False),
sa.Column(
'status',
# Dialect correction: a cliff calculation has a simulation run's
# states, so the column reuses the baseline's v2_simulation_status
# type instead of creating it again, which would fail.
postgresql.ENUM(
'pending',
'running',
'succeeded',
'failed',
name='v2_simulation_status',
create_type=False,
),
nullable=False,
),
sa.Column('modal_call_id', sqlmodel.sql.sqltypes.AutoString(length=255), nullable=True),
sa.Column('error_message', sqlmodel.sql.sqltypes.AutoString(), nullable=True),
sa.Column('started_at', sa.DateTime(timezone=True), nullable=True),
sa.Column('completed_at', sa.DateTime(timezone=True), nullable=True),
sa.Column('cliff_gap', sa.Float(), nullable=True),
sa.Column('cliff_share', sa.Float(), nullable=True),
sa.CheckConstraint("(cliff_gap IS NULL) = (cliff_share IS NULL) AND (status = 'succeeded') = (cliff_gap IS NOT NULL)", name=op.f('ck_cliff_impacts_result_when_succeeded')),
sa.CheckConstraint("status NOT IN ('succeeded', 'failed') OR completed_at IS NOT NULL", name=op.f('ck_cliff_impacts_finished_completion')),
sa.ForeignKeyConstraint(['simulation_run_id'], ['simulation_runs.id'], name=op.f('fk_cliff_impacts_simulation_run_id_simulation_runs'), ondelete='CASCADE'),
sa.PrimaryKeyConstraint('id', name=op.f('pk_cliff_impacts'))
)
op.create_index(op.f('ix_cliff_impacts_simulation_run_id'), 'cliff_impacts', ['simulation_run_id'], unique=False)
op.create_index('ix_cliff_impacts_status_started_at', 'cliff_impacts', ['status', 'started_at'], unique=False)
op.create_index('uq_cliff_impacts_unfailed', 'cliff_impacts', ['simulation_run_id'], unique=True, postgresql_where=sa.text("status <> 'failed'"), sqlite_where=sa.text("status <> 'failed'"))
# ### end Alembic commands ###


def downgrade() -> None:
# ### commands auto generated by Alembic - please adjust! ###
op.drop_index('uq_cliff_impacts_unfailed', table_name='cliff_impacts', postgresql_where=sa.text("status <> 'failed'"), sqlite_where=sa.text("status <> 'failed'"))
op.drop_index('ix_cliff_impacts_status_started_at', table_name='cliff_impacts')
op.drop_index(op.f('ix_cliff_impacts_simulation_run_id'), table_name='cliff_impacts')
op.drop_table('cliff_impacts')
# ### end Alembic commands ###
2 changes: 1 addition & 1 deletion policyengine_api/data/v2/migration_target.py
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@
from collections.abc import Mapping

V2_ALEMBIC_DISPOSABLE_TEST = "V2_ALEMBIC_DISPOSABLE_TEST"
V2_ALEMBIC_HEAD_REVISION = "f519a77ba122"
V2_ALEMBIC_HEAD_REVISION = "bc334f9877b7"
DISPOSABLE_DATABASE_NAME = "policyengine_v2_alembic_test"
DISPOSABLE_HOSTS = frozenset({"localhost", "127.0.0.1", "::1", "postgres"})
MIGRATION_ROLE = "policyengine_v2_migrator"
Expand Down
2 changes: 2 additions & 0 deletions policyengine_api/data/v2/models/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -78,6 +78,7 @@
AggregateType,
BudgetSummary,
ChangeAggregate,
CliffImpact,
CongressionalDistrictImpact,
ConstituencyImpact,
DecileImpact,
Expand Down Expand Up @@ -124,6 +125,7 @@
"CatalogEntity",
"CatalogEnumValue",
"ChangeAggregate",
"CliffImpact",
"CongressionalDistrictImpact",
"ConstituencyImpact",
"Country",
Expand Down
88 changes: 84 additions & 4 deletions policyengine_api/data/v2/models/run_results.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,20 +2,28 @@

Results that compare a report run's baseline and reform belong to the report
run and name its two simulation runs; each such row is unique within its report
run. Poverty and inequality, which describe one simulation, belong to a
simulation run.
run. Poverty, inequality and cliff impacts, which describe one simulation,
belong to a simulation run.
"""

from datetime import datetime
from enum import StrEnum
from typing import Any
from uuid import UUID

import sqlalchemy as sa
from sqlmodel import Field, Relationship

from policyengine_api.data.v2.models.base import IdentifiedModel, enum_type
from policyengine_api.data.v2.models.base import (
IdentifiedModel,
TimestampedModel,
enum_type,
)
from policyengine_api.data.v2.models.reports import ReportRun
from policyengine_api.data.v2.models.simulations import SimulationRun
from policyengine_api.data.v2.models.simulations import (
SimulationRun,
SimulationStatus,
)


class OutputStatus(StrEnum):
Expand Down Expand Up @@ -401,6 +409,78 @@ class Poverty(IdentifiedModel, table=True):
simulation_run: SimulationRun = Relationship(back_populates="poverty_results")


# The cliff calculations a new request may join, of which a simulation run has
# at most one: a failed calculation never counts.
UNFAILED_CLIFF_CALCULATIONS = "status <> 'failed'"


class CliffImpact(TimestampedModel, table=True):
"""One cliff calculation of a simulation run, and its result once it succeeds.

``cliff_gap`` is the weighted total of the net income people would lose
from a small rise in their earnings, and ``cliff_share`` the weighted
number of people on a cliff divided by the weighted number of adults.
They need marginal tax rates, which recalculate each household with extra
earnings, so a run's cliffs are calculated apart from the run, on its
dataset, policy and dynamic, only when a request asks for them, and only
for a run on the current versions. The result has no versions of its own:
it inherits its run's, and every report naming the run shares it.

The simulation service inserts a pending calculation and records its
Modal call, or fails it when Modal refuses the call; the Modal worker
claims it under its call, then writes the two numbers with ``succeeded``,
or ``failed`` with its ``code: message`` error. A run has at most one
calculation that hasn't failed, which identical requests join; a failed
calculation never counts, and the next request starts another. Deleting
the run deletes its calculations.
"""

__tablename__ = "cliff_impacts"
__table_args__ = (
sa.CheckConstraint(
"status NOT IN ('succeeded', 'failed') OR completed_at IS NOT NULL",
name="ck_cliff_impacts_finished_completion",
),
sa.CheckConstraint(
"(cliff_gap IS NULL) = (cliff_share IS NULL) "
"AND (status = 'succeeded') = (cliff_gap IS NOT NULL)",
name="ck_cliff_impacts_result_when_succeeded",
),
sa.Index("ix_cliff_impacts_status_started_at", "status", "started_at"),
sa.Index(
"uq_cliff_impacts_unfailed",
"simulation_run_id",
unique=True,
postgresql_where=sa.text(UNFAILED_CLIFF_CALCULATIONS),
sqlite_where=sa.text(UNFAILED_CLIFF_CALCULATIONS),
),
)

simulation_run_id: UUID = Field(
foreign_key="simulation_runs.id",
ondelete="CASCADE",
index=True,
)
status: SimulationStatus = Field(
default=SimulationStatus.PENDING,
sa_type=enum_type(SimulationStatus, "v2_simulation_status"),
)
modal_call_id: str | None = Field(default=None, max_length=255)
error_message: str | None = None
started_at: datetime | None = Field(
default=None,
sa_type=sa.DateTime(timezone=True),
)
completed_at: datetime | None = Field(
default=None,
sa_type=sa.DateTime(timezone=True),
)
cliff_gap: float | None = None
cliff_share: float | None = None

simulation_run: SimulationRun = Relationship(back_populates="cliff_impacts")


class ProgramStatistics(IdentifiedModel, table=True):
"""Baseline and reform totals, counts, winners and losers for one program.

Expand Down
Loading
Loading