Skip to content

ci: test shared code changes in PR checks - #352

Draft
tnazarew wants to merge 1 commit into
mainfrom
ci/pr-coverage
Draft

tnazarew wants to merge 1 commit into
mainfrom
ci/pr-coverage

Conversation

@tnazarew

@tnazarew tnazarew commented Nov 5, 2026

Copy link
Copy Markdown
Collaborator

Why

PR checks decide what to run only from producer/<p>/ and consumer/… paths. A change to the validator (scripts/validate_ol_events.py, report.py, compare_events.py), a producer workflow (.github/workflows/producer_*.yml) or a shared action (dataproc_cluster, run_event_validation) merges without the affected component ever running in CI.

What

  • scripts/get_pr_components.py (tested) replaces the inline path check in main_pr.yml:
    Changed PR check does
    a component's files or its workflow real run of that component
    dataproc_cluster, get_openlineage_artifacts, upload_artifacts actions, get_valid_test_scenarios.sh real run of the producers using them
    validation code (validator, report, compare scripts, run_event_validation, collect workflow) real run of dbt and scenarios (cheap, no cloud), dataplex for report.py/compare_events.py; re-validation of stored events for spark_dataproc and hive_dataproc (no clusters)
    anything else nothing, as before
  • Baseline events: the release check passes update-baseline: true to the Spark and Hive workflows, which then store each validated pair's events (and Spark's status.json) under gs://open-lineage-e2e/baseline-events/<component>/<cv>/<ol>/. Every other caller leaves it off. The bucket's 90-day delete rule only covers events/ and logs/.
  • revalidate_stored_events.yml: one job per component validates every baseline pair with the PR's code and uploads one report artifact; collect-and-compare-reports compares it with report.json like any other report, so new failures fail the PR. No baseline → a warning ("not covered"), not a failure.
  • The spec fetch moves from the run_event_validation action into dev/validation/get_ol_specs.sh (same behaviour), shared with the re-validation.
  • pytest setup (scripts/tests/, scripts/requirements-dev.txt) and a script-tests job.

Verification

  • pytest scripts/tests → 25 passed (path map cases, incl. the old paths keeping their behaviour and anchored prefixes).
  • Local smoke test of the re-validation steps on stored Hive events: specs fetched for 1.34.0 and 1.55.0 through get_ol_specs.sh, both pairs validated, one merged report keyed like the release entries.
  • actionlint: no new findings apart from quoting notes in pre-existing steps.
  • This PR changes the Spark and Hive workflows, so its own check runs both for real.

After merge

The baseline is empty until the next release run that runs Spark/Hive. I'll seed it once (with approval) by copying the last release run's stored Spark events and the latest Hive events into baseline-events/, so the next validator PR is covered right away.

Note: Hive currently fails its semantic checks, but no Hive report ever reaches the comparison (an upload path bug), so that isn't visible yet. A follow-up PR fixes the upload and the stale Hive expectations.

🤖 Generated with Claude Code

PR checks picked components only from producer/<p>/ and consumer/ paths, so
changes to the validator, report code, producer workflows or shared actions
merged without the affected component ever running.

- scripts/get_pr_components.py maps changed files to components: a
  component's files, its workflow or a shared action/script it uses run it
  for real; validation code changes run dbt and scenarios for real and
  re-validate the stored events of the last release run for spark_dataproc
  and hive_dataproc (no clusters).
- The release check stores each validated Spark/Hive pair's events under
  gs://open-lineage-e2e/baseline-events/ (update-baseline input, off for
  every other caller).
- revalidate_stored_events.yml validates that baseline with the PR's code,
  one report artifact per component, compared like any other report.
- The spec fetch moves from the run_event_validation action into
  dev/validation/get_ol_specs.sh, shared with the re-validation.
- Adds pytest setup and a script-tests job.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: tnazarew <tomasz.nazarewicz@getindata.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant