Repository navigation
Don't hang worker shutdown when a Nexus task is cancelled while encoding its failure - #2020
Merged
tconley1428 merged 2 commits intoOct 26, 2026
Conversation
tconley1428
reviewed
Oct 26, 2026
tconley1428
requested changes
Oct 26, 2026
tconley1428
left a comment
Contributor
There was a problem hiding this comment.
Raw sleep should be removed before completion.
tconley1428
approved these changes
Oct 26, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
When a Nexus start or cancel handler fails, the failure is encoded inside the
except BaseExceptionblock. If the task gets cancelled during that encoding, for example, when the worker shuts down, theCancelledErrorescapes the block. The outerexcept Exceptiondoesn't catch it, so_complete_task()is never called and core never gets a completion for the task. Worker shutdown then waits for it forever, which is what the comment on_complete_task()warns about.Now a cancel during that encode sends an
ack_cancelcompletion, the same thing a cancel during the success path's encode already does.Added a test with a payload codec that is slow to encode, so worker shutdown cancels the task while it's encoding the failure. It covers the start handler and the cancel handler. On main, shutdown doesn't finish within 10 seconds in either case. With the fix, both pass. The rest of
tests/nexusgives the same results with and without this change.