Repository navigation
Conversation
|
Here is the summary of changes. You are about to add 1 region tag.
This comment is generated by snippet-bot.
|
There was a problem hiding this comment.
Code Review
This pull request introduces a Cloud Function to deduplicate Cloud Scheduler job requests using Firestore for distributed locking and lease management. The review feedback highlights a potential race condition where a slow worker whose lease has expired could inadvertently overwrite or delete a lease claimed by a subsequent retry. To address this, the reviewer suggests using the lease expiration timestamp as a fencing token and performing both the completion and release operations transactionally, providing concrete code suggestions for the implementation and corresponding test updates.
XrossFox
left a comment
There was a problem hiding this comment.
The code LGTM, but i left a comment.
…ob requests Add cloudscheduler_deduplicate_job_requests, a Cloud Run function that does a Cloud Scheduler job's work once per scheduled run, even when Cloud Scheduler delivers the same run more than once. The function identifies a run by the job name and the X-CloudScheduler-ScheduleTime header, which stays the same across retries. It claims the run in a Firestore transaction with a lease: - A completed run is skipped. - An active claim returns 409. - An expired claim from a crashed attempt can be claimed again. If the work fails, the function releases the claim so that Cloud Scheduler's retry can process the run. This rewrites the sample proposed in #14702 and keeps its region tag. Unit tests use an in-memory fake Firestore. Integration tests run against a real Firestore database and are skipped unless RUN_INTEGRATION_TESTS is set. Co-authored-by: Camie Kim <12733285+camiekim@users.noreply.github.com>
Address PR review feedback: if an attempt's lease expired and a retry claimed the run, the slow attempt could still delete or complete the newer claim. Each attempt now records a unique attempt ID in its claim. A single transactional finish_run() helper completes or releases the run only if the record still holds that attempt's ID. Add unit and integration tests for a stale attempt that fails or succeeds after a retry claimed the run.
94bf9df to
08cbe95
Compare
Description
Adds
cloudscheduler_deduplicate_job_requests, a Cloud Run function that does a Cloud Scheduler job's work once per scheduled run, even when Cloud Scheduler delivers the same run more than once. This rewrites #14702 by @camiekim, who is credited as co-author, and keeps that PR's region tag so the in-progress "Deduplicate job requests" docs page doesn't need to change.How it works
X-CloudScheduler-ScheduleTime, hashed into a Firestore document ID. The schedule time stays the same across retries, so the header is used exactly as received.expire_atfield for a Firestore TTL policy.Changes from #14702
cloud_scheduler/deduplicate_job_requests/, as a functions-framework function with its own requirements (google-cloud-firestore2.34.1), instead of adding Flask and Firestore tosnippets/.Scope. In testing, HTTP targets received
X-CloudScheduler-JobName(the job ID) andX-CloudScheduler-ScheduleTime. The API reference documents the same headers for App Engine targets, so this sample covers both. Pub/Sub messages from Cloud Scheduler carry no run identifier, and Google API targets such as Workflows have no user handler, so those targets are better covered on the docs page. A forced run sends the next scheduled time, so header-based deduplication then skips that scheduled run. A comment in the sample notes this.Tests (in
tests/)RUN_INTEGRATION_TESTS=true; setFIRESTORE_DATABASEto use a named database.Checklist
Testing
Compliance & Style
cloud_scheduler, which the default owners cover.Post-Approval Actions