Repository navigation
Conversation
Signed-off-by: Jesse Hodges <hodges.jesse@gmail.com>
…ecks Signed-off-by: Jesse Hodges <hodges.jesse@gmail.com>
Signed-off-by: Jesse Hodges <hodges.jesse@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Related to #5825.
A production plan can finish its planned backfill and then fail the no-gap check because a scheduled run advanced the incumbent snapshot while the replacement was being built. This adds an opt-in policy that allows promotion and finalization when the replacement is missing only a bounded, contiguous trailing suffix created after the plan was built. The suffix stays missing in interval state so the normal scheduler can repair it.
Strict no-gap behavior remains the default (
0). The GitHub bot can opt in withmax_trailing_gap_intervals_on_deploy_to_prod=1; Python callers can useContext.plan_builder(..., no_gaps=True, max_trailing_gap_intervals=1). The underlying plan/promotion policy is independent of GitHub. This PR does not add a CLI flag or notebook magic option.Safety and scope
This is a bounded freshness tradeoff rather than the model-level locking proposed in the issue. Subsequent scheduler selections must include upstream dependencies. The bound applies to drift observed by the promotion check; an already-running scheduler is not fenced from advancing the old snapshot afterward. This change does not add durable repair tracking.
Validation
make stylepassed: Ruff, Ruff formatting, mypy, and migration validation.The implementation has also run in our production fork. Two production deployment attempts successfully used the allowance. For one, operations metadata and retained interval state corroborate subsequent repair on the exact promoted snapshot: the covering scheduler batch completed about 4m18s after finalization. This verifies one model's repair; it does not establish all downstream row values or a durable repair-time guarantee.