Repository navigation
Take data_source on /v2/reports, answer each report's dataset_independent_key, and find reports by it - #4310
Merged
Merged
Conversation
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…y it (failing) Every report read answers reports.dataset_independent_key, and GET /v2/reports takes dataset_independent_key in place of report_id to find the country's reports that share it, newest first. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Every report item, of every kind, answers reports.dataset_independent_key as the simulation service stored it, null for a report stored before the keys. GET /v2/reports takes dataset_independent_key in place of report_id and lists the country's reports that share it, newest created first, at most the 25 created last, through ix_reports_dataset_independent_key. The query takes one of the two. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…e (failing) POST /v2/reports takes data_source: "local_area_dataset" on an economy report and sends it on only when given; null, another value and the field on a budget window or household report are refused before the simulation service is asked. The simulation service checks the region, the country and the release: its 422 answers 422, and its 400 for its own local-area dataset state 503. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
EconomyReportFields takes data_source: "local_area_dataset", left out unless given, as spm is; an explicit null or another value is refused with 422, and budget windows and household reports refuse the field as unknown. The report request sends it only when given. The simulation service checks the region, the country and the release, and its refusals map as before: a 422 answers 422, and a 400 about its own local-area dataset 503. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… into v2plan/c1-30-data-source-and-keys
A UK-wide or nation report whose dataset is the one the catalog version gives both the constituency and local-authority regions is a local-area report: its read says data_source local_area_dataset, and its result lists each level's withheld area codes, sorted, with a level whose areas were all withheld read as empty. Every other result leaves the two lists out, as the simulation service's does. The result fixture stores withheld_area_codes as PolicyEngine/policyengine-sim-api#1068's writer does. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A UK-wide or nation report whose dataset is the default dataset of both the constituency and the local-authority regions of its catalog version is a local-area report, as the simulation service decides which dataset local areas use. Its read says data_source local_area_dataset, its result lists each level's withheld area codes from withheld_area_codes, sorted, and a level whose areas were all withheld reads as empty. Every other report reads data_source null, and its result leaves the two lists out, as the simulation service's SingleYearMacroOutput does. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The two report items' docstrings keep both sides: the null label and the dataset-independent key. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The request's ReportDataSource and its LOCAL_AREA_DATASET value now name the read's data_source too. The list result's docstring gives the search's order, and the read's data_source says it comes from the report's region and dataset. The stand-in refusals match the simulation service's answers, the two SQLite read tests share their setup (tests.unit.v2.sqlite_report_rows, which drops one suppression), and PostgreSQL now reads a UK-wide report on the national dataset and a local-area report with an empty area level. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The simulation service's two data_source refusals now answer the UK requests that draw them: a UK-wide report on a release without a local-area dataset, and a constituency's own report. The value's comment says a local area's own report never reads it, and the economy item's docstring reads in order. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #4310 +/- ##
=======================================
Coverage 93.67% 93.67%
=======================================
Files 397 397
Lines 22831 22885 +54
Branches 3589 3599 +10
=======================================
+ Hits 21387 21438 +51
- Misses 840 845 +5
+ Partials 604 602 -2
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Resolves the overlap the two tasks agreed C1 · 30 resolves: the search by dataset_independent_key takes the list route's optional user credential and reads each report through the same requester view, so a search shows the user's saved label and newest recorded run as a list does, and refuses a browser credential naming a signed-in user with a 401. The route's list or search choice moves into a helper to keep the router builder within ruff's complexity limit. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This branch was previously deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #4309
Summary
Builds on #4303 (merged 2026-11-16,
e96e52ffb; it addsreports.dataset_independent_keyandix_reports_dataset_independent_key), and on #4307 (merged 2026-11-17,c5b8aab32): #4307 stops the Public API sendingdataset_nameto the simulation service and makesPOST /v2/reportsandPOST /v2/simulationsrefuse it, so this PR adds nodataset_namechange of its own. #4313 (per-user report runs, merged 2026-11-17,05332d347) merged first, so this PR mergedmasterand resolved the overlap: the search bydataset_independent_keytakes the same optional user credential as the list and reads each report the same way (the user's saved label and description, the run of their newest request, and a 401 for a browser credential that names a signed-in user).data_sourceonPOST /v2/reports. An economy report takesdata_source: "local_area_dataset", left out unless given (asspm): an explicit null or another value is refused with 422, and budget windows and household reports refuse the field as unknown. It is sent to the simulation service only when given. The Public API checks only the value; the simulation service checks the country (uk), the region (ukand the fourcountry/*nations) and whether the release gives local areas a dataset of their own. Its 422s answer 422 with its code and reasons, and its 400local_areas_unsupportedandlocal_area_dataset_mismatch(its own state) answer 503, through the existing client mapping. The value is defined once (ReportDataSource,LOCAL_AREA_DATASET).dataset_independent_keyon every report read. Every report item of every kind (detail, list, search, and the create and re-run answers) answers the storedreports.dataset_independent_key, null for a report stored before the keys. The Public API never computes a key.GET /v2/reports?country_id=…&dataset_independent_key=…. The list route takes eitherreport_id(repeated, at most 25, as before) or onedataset_independent_key(64 lowercase hexadecimal digits), never both or neither (422). It answers the country's reports with that key as summary items, newest created first (ties to the larger ID), at most the 25 created last, throughix_reports_dataset_independent_key. No credential, as every report read; the runtime role's existingSELECTonreports.data_source. Its read saysdata_source: "local_area_dataset"(null for every other economy report), and its result addsconstituency_withheld_codesandlocal_authority_withheld_codesfromwithheld_area_codes(Record the UK area codes disclosure control withholds from a report run #4305), sorted,[]when none were withheld; an area level whose areas were all withheld reads[]. Every other result leaves the two lists out, as the simulation service'sSingleYearMacroOutputdoes (Calculate UK-wide and nation reports on the local-area dataset, with their areas and withheld codes policyengine-sim-api#1068). The withheld codes are read only for a local-area report.What must merge and deploy first:
94dc43079186deployed before this) and Stop sending dataset_name to the simulation service, and refuse it on /v2 #4307 (merged; in production before the simulation service removesdataset_name);/reportstakesdata_source, stacked on Ensure get_household_year doesn't break on empty age variable #1068). Until C1 · 29a deploys, the simulation service refusesdata_sourceas an unknown field and this route answers that 422; until C1 · 27 deploys, reports have no keys and the search finds none. The withheld codes are written by Calculate UK-wide and nation reports on the local-area dataset, with their areas and withheld codes policyengine-sim-api#1068's writer, merged to the simulation service'smainon 2026-11-16 (9ea2b48ce), with the sameconstituency_withheld_codesandlocal_authority_withheld_codesfields this PR reads.Migration: no route group moves;
/v2/reportsstays on FastAPI. User-visible contract changes: the new request field, the new query parameter (andreport_idbecoming optional when the key is given), and the new read fields. The generated migration contract documents are unchanged by the export.Inline suppression added:
# noqa: PLR0913onstore_reportintests/integration/stored_report_rows.py(the connection and catalog, then each column a test varies): it gains two keyword arguments for the key and the creation time, and grouping its existing arguments would change every caller.Testing
Each change was built test first (the failing tests are their own commits). Run by node ID only:
tests/unit/v2/test_report_dataset_independent_keys.py(SQLite: every kind of read answers the key; the search's country, order, ties, limit and empty answer),tests/unit/v2/test_local_area_report_reads.py(SQLite: which reports read as local-area reports),tests/unit/v2/test_economy_result_rebuild.py(withheld codes rebuilt sorted; an empty level; other results leave the lists out),tests/unit/v2/test_report_read_routes.py(the key, the search route, its refused queries, OpenAPI),tests/unit/v2/test_report_create_routes.pyandtests/unit/libs/test_simulation_reports.py(data_sourcepassed on and refused; the simulation service's refusals, as stand-ins shaped as its C1 · 29a answers);tests/integration/test_v2_report_reads.py): the search; a local-area report's areas and withheld codes; an empty level; UK-wide and nation reports on the national dataset;lint-imports,python -m scripts.rules validate,python scripts/run_quality_guards.py, ruff, ruff format, mypy, andpython scripts/export_migration_contracts.py(no change). No revision or grant change.Two read-only review rounds: round 1 found no bugs (documentation, one shared value, test setup and coverage fixed); round 2, on those fixes, found no bugs (test and documentation fixes only).
🤖 Generated with Claude Code