Skip to content

fix(supervisor): report isolation backend Attach refusals to the gateway - #4791

Open
shiju-nv wants to merge 1 commit into
NVIDIA:mainfrom
shiju-nv:fix/4739-attach-refusal-report/shiju-nv
Open

shiju-nv wants to merge 1 commit into
NVIDIA:mainfrom
shiju-nv:fix/4739-attach-refusal-report/shiju-nv

Conversation

@shiju-nv

Copy link
Copy Markdown
Collaborator

Summary

When an isolation backend refused Attach, the supervisor exited without telling the gateway. The configuration stayed ConfigurationReady: True (ConfigurationAccepted), and a revision the supervisor had written at startup stayed Pending with no error. The supervisor now reports the refusal: the gateway records the accepted configuration as rejected, and a revision this start wrote ends Failed with the backend's message.

Related Issue

Closes #4739

Changes

  • proto/openshell.proto: ReportSandboxConfigurationRequest gains bool refused_before_start = 4. It is set only with a Rejected admission for the generation that instance already reported Accepted. Omitted or false keeps today's behavior.
  • crates/openshell-server/src/grpc/policy.rs: handle_report_sandbox_configuration moves an Accepted configuration to Rejected only when the flag is set and the report names the same instance and generation. The message is gateway-authored; runtime text stays out of public status, as before. The flag with any other state is INVALID_ARGUMENT. Every unflagged report behaves as before, including a runtime that fails to apply the generation it is running (for example unreachable middleware), which keeps the accepted state.
  • crates/openshell-core/src/grpc_client.rs: report_sandbox_configuration delegates to the new report_sandbox_configuration_request, which sends a caller-built request. The span name is unchanged.
  • crates/openshell-supervisor/src/backend_setup.rs: attach tags Denied and Attach errors as refusals. Descriptor (driver and TLS faults) and Unavailable are not tagged.
  • crates/openshell-supervisor/src/lib.rs: on a tagged Attach error, run_sandbox_with_backend sends the flagged Rejected report for the accepted generation (one bounded attempt), then reports a revision this start wrote (image discovery or baseline-path write-back) Failed with the backend's message. A revision the user wrote is not marked. The original error is still returned.
  • crates/openshell-server/src/storage_proto.rs: PUBLIC_RPC_SCHEMA_SHA256 for the new public-only field, taken from the failing test output. Durable schemas are unchanged.
  • sdk/go/proto/openshellv1/openshell.pb.go: regenerated with mise run go:proto:gen.
  • docs/extensibility/isolation-backends.mdx: what users see after a refusal.

Testing

  • cargo test -p openshell-supervisor --lib (177 passed), cargo test -p openshell-server --lib (2338 passed, 21 ignored, including public_and_durable_schema_inventories_are_complete), cargo test -p openshell-core --lib (574 passed)
  • cargo clippy -p openshell-supervisor -p openshell-server -p openshell-core --all-targets -- -D warnings, cargo fmt --check, go build ./... in sdk/go, mise run go:proto:check
  • Unit tests added:
    • attach_refusal_is_marked_as_a_refusal: a refused Attach carries the refusal code.
    • attach_refusal_reports_the_accepted_generation_as_refused: the report names the accepted instance and generation and sets refused_before_start; transient errors, local policy and a missing instance send nothing.
    • attach_refusal_fails_only_the_revision_this_start_wrote.
    • configuration_admission_records_refusal_of_accepted_generation: an unflagged same-generation rejection keeps Accepted, the flag with Accepted is INVALID_ARGUMENT, and a flagged rejection becomes ConfigurationInvalid without runtime text.
    • Each fails with its fix removed (checked by mutation).
  • E2E tests added/updated (if applicable): none. Checked by hand with examples/isolation-backend-v8, whose backend refuses Attach when process.run_as_user differs from the host identity:
    • with the policy run_as_user: "1500" on a uid 502 host, openshell sandbox get shows ConfigurationReady: False (ConfigurationInvalid) with the gateway message;
    • on main the same run shows ConfigurationReady: True (ConfigurationAccepted).

Recovery

Delete the sandbox and create it again with a policy the backend accepts. A refused sandbox stays in Provisioning until its repair deadline, and sandbox start, sandbox stop and a process-policy change are all refused in that phase, as on main.

Checklist

  • Follows Conventional Commits
  • Commits are signed off (DCO)
  • Architecture docs updated (if applicable)

When an isolation backend refused at Attach, the supervisor exited without
telling the gateway. The configuration it had reported Accepted before Attach
stayed accepted, so the sandbox showed ConfigurationReady: True, and the
revision it wrote at startup stayed Pending with no error.

ReportSandboxConfigurationRequest gains refused_before_start. On a Denied or
Attach refusal the supervisor reports the generation it accepted as Rejected
with the flag set, and the gateway records the configuration as rejected with
gateway-authored text. Unflagged reports keep today's behavior, so a runtime
that fails to apply the generation it is running stays accepted. The
supervisor also marks the revision this start wrote as Failed with the
backend's message. A revision the user wrote and transient errors are left
alone.

Closes NVIDIA#4739

Signed-off-by: Shiju <shiju@nvidia.com>
@github-actions

Copy link
Copy Markdown

Compatibility review (advisory)

New or updated breaking changes; see the full report for all findings.

1. rust-sdk — No explicit risk acceptance found

ReportSandboxConfigurationRequest gains refused_before_start, so its publicly re-exported prost Rust struct now has a fourth required struct-literal field instead of three.

  • Impact: Rust consumers constructing openshell_sdk::raw::proto::ReportSandboxConfigurationRequest { ... } without ..Default::default() fail to compile because refused_before_start is missing.
  • Action: Remove the field from v0.1.6 and introduce it in a minor release with migration guidance for Rust raw callers to add refused_before_start: false or ..Default::default().
  • PR: fix(supervisor): report isolation backend Attach refusals to the gateway #4791
Review details
  • Execution: complete
  • Candidate: PR #4791 (ad0916e8d87c8ebd32c53bac3dca78208b750ca8)
  • Baseline: d22394370d2d8b27859ffa772cac3aaaa79feb2c (d22394370d2d8b27859ffa772cac3aaaa79feb2c)
  • Reviewed: rust-sdk, python-sdk, go-sdk, typescript-sdk, api-behavior, cli, configuration, policy, helm, state

Advisory review; publication gates are unchanged.

Branch check run

@shiju-nv
shiju-nv force-pushed the fix/4739-attach-refusal-report/shiju-nv branch from 085e666 to 882780f Compare October 21, 2026 07:28

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

bug(supervisor): an isolation backend's Attach refusal is not reported to the gateway

1 participant