Skip to content

hygiene: warn about untested scenarios, require scenario descriptions - #355

Draft
tnazarew wants to merge 1 commit into
ci/pr-coveragefrom
hygiene/repo-cleanup
Draft

tnazarew wants to merge 1 commit into
ci/pr-coveragefrom
hygiene/repo-cleanup

Conversation

@tnazarew

@tnazarew tnazarew commented Nov 5, 2026

Copy link
Copy Markdown
Collaborator

Stacked on #352 (base ci/pr-coverage, for the pytest setup); retarget to main after #352 merges.

What

  • scripts/check_scenario_coverage.py (warnings only, exits 0), run in the PR check's structure step and in the release check's initialize job (job summary). The tested matrix is versions.json plus the latest OpenLineage release and, when releases.json tracks it, the latest component release. It reports:
    • scenarios whose version ranges match no tested version. Today: spark_dataproc/bigtable, capped at OpenLineage 1.0.0, so it's never tested.
    • tested component versions no scenario covers. Today: dbt 1.12.6, the release the release check tracks. csv_to_postgres is gated to dbt 1.8.0, so every "new dbt release" run tests nothing (both steps skipped in run 41619028964).
  • check_structure.sh requires scenario.md to be non-empty (consumer, dataplex and producer scenarios). Fills the two empty ones: consumer/scenarios/spark_dataproc_bigquery_shakespare (described from its events) and the producer template.
  • .gitignore: .DS_Store, *.jar, dbt's .user.yml. Removes the tracked producer/dbt/runner/.user.yml (dbt's per-machine anonymous id).
  • Removes generated-files/historical_reports/: nothing ever wrote to it (collect never uploads updated-new-report.json). report.json's git history already holds every run's report.

The stricter run-script checks (non-empty, executable, bash -n) come with the PR that makes the empty run_hive_dataproc_tests.sh a working script, so this PR's own check stays green.

Verification

  • pytest scripts/tests → 34 passed: coverage (never-running scenario, uncovered version, latest OL counts, CLI on the real repo), structure check on a copy of the repo tree (passes; empty scenario.md in producer/consumer/dataplex scenarios fails; missing still fails). The empty-scenario.md cases failed before the change.
  • ./scripts/check_structure.sh → exit 0; python3 scripts/check_scenario_coverage.py → the two warnings above.
  • actionlint: no new findings.

🤖 Generated with Claude Code

- scripts/check_scenario_coverage.py warns (PR check and release run
  summary) about scenarios whose version ranges match no tested version
  (today: spark_dataproc/bigtable, capped at OpenLineage 1.0.0) and tested
  component versions no scenario covers (today: dbt 1.12.6, the release
  the release check tracks; csv_to_postgres is gated to 1.8.0, so those
  runs test nothing).
- check_structure.sh requires scenario.md to be non-empty; fills the two
  empty ones (spark_dataproc_bigquery_shakespare, the producer template).
- .gitignore: .DS_Store, *.jar, dbt's .user.yml; removes the tracked
  producer/dbt/runner/.user.yml (a per-machine anonymous id).
- Removes generated-files/historical_reports/: nothing ever wrote to it
  (collect never uploads updated-new-report.json); report.json's git
  history already holds every run's report.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: tnazarew <tomasz.nazarewicz@getindata.com>
@tnazarew
tnazarew force-pushed the hygiene/repo-cleanup branch from 0e5e30f to e45ba4e Compare November 12, 2026 13:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant