Skip to content

test(oauth): check OAuth scopes against a live server - #1188

Merged
EhabY merged 6 commits into
mainfrom
test/vsc-24-oauth-scope-suite
Nov 13, 2026
Merged

EhabY merged 6 commits into
mainfrom
test/vsc-24-oauth-scope-suite

Conversation

@EhabY

@EhabY EhabY commented Nov 12, 2026 •

Copy link
Copy Markdown
Collaborator

Part of VSC-24. Follows #1138, which changed the scopes the extension requests. This PR checks them against a live server.

Why

Released servers ignore requested scopes, so unit tests can't show that the extension requests enough. A missing scope shows up only against a server that enforces scopes, and sometimes only as an empty list, a closed socket, or a CLI warning.

What catches what

  • pnpm test, every PR, about 1.5 seconds: test/unit/oauthScopeProbes.test.ts uses the TypeScript checker to find every CoderApi request or stream method that src/ references, including through destructuring and Pick<CoderApi, ...>. It fails when a method has no probe and no listed reason in unprobedMethods, or when either list is stale.
  • pnpm test:scopes, on PRs that touch the code the probes run, about 50 seconds:
    • Signs a member in and refreshes the token through the extension's own OAuth code. Both must grant exactly the scopes in the extension's authorize URL.
    • Runs every probe with the refreshed token. List probes require results, since the server drops rows the token can't read. The server must accept every stream, and streams that authorize each message must deliver one.
    • Sends the requests coder ssh makes for an owned and a shared workspace, and runs the one-build update and coder start through the extension's updateWorkspace and startWorkspace.
  • Nightly, against coder-preview:latest: catches server changes before a release, and reports failures in a "Nightly OAuth scope test failed" issue.

PRs use v2.38.0, pinned by digest: the first release that enforces requested scopes.

docker compose -f test/scopes/compose.yaml up -d --wait --force-recreate --renew-anon-volumes
CODER_SCOPES_TEST_URL=http://localhost:7080 pnpm test:scopes

Where the lines go

+1089/-4. Almost all of it is new test code; src/ changes only by exporting rawDataToString. Start with probes.ts: it lists what the suite checks, and everything under support/ exists to run it.

Lines File What
112 test/scopes/probes.ts What gets checked: one probe per API method, the CLI probes, and the methods that need none
51 test/scopes/oauthScopes.test.ts The suite: scope grants, then each probe
84 test/scopes/support/deployment.ts The shared setup: users, a template, two workspaces and the OAuth session
102 test/scopes/support/oauthLogin.ts The extension's OAuth login and refresh, with the browser consent stubbed
183 test/scopes/support/workspaces.ts, template.tf Template, workspaces and waiting on builds through the typed API
115 test/scopes/support/workspaceActions.ts coder ssh requests, the one-build update, and coder start with the CLI the server serves
71 test/scopes/support/streams.ts Stream probes: the server accepts the stream, or it delivers a message
81 test/scopes/support/server.ts REST clients, sign-in and sockets
143 test/unit/oauthScopeProbes.test.ts The source scan that keeps probes.ts complete
63 .github/workflows/oauth-scopes.yaml PR and nightly runs
52 test/scopes/compose.yaml, test/scopes/vitest.config.mts The throwaway deployment and the suite's own Vitest config
31 Docs, package.json, pnpm-workspace.yaml docs/TESTING.md section, test:scopes script

Not covered

🤖 Generated with Claude Code

@linear-code

linear-code Bot commented Nov 12, 2026 •

Copy link
Copy Markdown

VSC-24

VSC-47

@EhabY
EhabY force-pushed the test/vsc-24-oauth-scope-suite branch 4 times, most recently from f4df09b to c02b81e Compare November 12, 2026 15:07
Signs a member in and refreshes the token through the extension's OAuth
code against a server that enforces scopes, then probes every Coder API
request and stream the extension makes, plus the requests `coder ssh` and
`coder start` send. A unit test scans src/ with the TypeScript checker and
fails when a CoderApi method the extension calls has no probe.

The live suite has its own Vitest config, so `pnpm test` leaves it out. It
runs on PRs against the pinned v2.38.0 release and nightly against
coder-preview:latest.
@EhabY
EhabY force-pushed the test/vsc-24-oauth-scope-suite branch from c02b81e to ddef224 Compare November 13, 2026 15:20
@EhabY
EhabY requested a review from jeremyruppel November 13, 2026 15:20
Comment thread test/scopes/support/server.ts Resolved
@EhabY
EhabY requested a review from jeremyruppel November 13, 2026 16:13

@jeremyruppel jeremyruppel left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for this, the scan in oauthScopeProbes.test.ts is a nice way to keep the probes complete. There's one bug that makes three stream probes always pass, plus two small things inline.

Drafted with Coder Agents.

Comment thread test/scopes/support/streams.ts Resolved
Comment thread package.json Resolved
Comment thread .github/workflows/oauth-scopes.yaml Resolved
Comment thread test/scopes/support/deployment.ts Resolved
@EhabY
EhabY requested a review from jeremyruppel November 13, 2026 18:00
@EhabY
EhabY merged commit 5969dc3 into main Nov 13, 2026
13 checks passed
@EhabY
EhabY deleted the test/vsc-24-oauth-scope-suite branch November 13, 2026 18:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants