Skip to content

feat(cli): render a resource's drift as a construct diff in checkly/codegen [RED-1547] - #1612

Open
sorccu wants to merge 6 commits into
mainfrom
simo/red-1547-codegen-diff-options
Open

sorccu wants to merge 6 commits into
mainfrom
simo/red-1547-codegen-diff-options

Conversation

@sorccu

@sorccu sorccu commented Nov 17, 2026 •

Copy link
Copy Markdown
Member

Affected Components

  • CLI
  • Create CLI
  • Test
  • Docs
  • Examples
  • Other

Notes for the Reviewer

Adds generateConstructDiff to the experimental checkly/codegen entry (RED-1547). Given a resource as Checkly holds it now (import format), the resource as its last deploy left it (deployed, in the deploy payload's vocabulary) and the changes the deploy diff reports between the two, it renders deployed (−) → current (+) as a construct diff with the import codegen, the way checkly deploy's preview shows a construct diff. Checkly's backend will use it to show drift as code in the web app.

Commits.

  1. A refactor that moves the preview's construct diff core out of render.ts into construct-diff.ts. The deploy preview's output is unchanged; its specs pass without edits.
  2. The export: codegen/diff.ts (the public function and GenerateDiff* types) over deploy-diff/drift.ts.
  3. A fix for objects removed in Checkly (superseded by 4).
  4. The deployed side now comes from the deployed input instead of undoing each change on the current resource, which could not restore structure the current resource no longer holds.
  5. Without projectDefaults, the diff writes every property, as generateConstructFiles does, so it reads like the code generated for the resource; {} keeps the previous output.
  6. In the diff, a redaction rule without a condition (a password, a header value) leaves an empty value as it is, so no credential the resource never had is shown; locked and secret variables stay masked. The deploy preview is unchanged.

How the deployed side is built (deployedSide).

  • A position no change reaches equals the current resource, so hashes, set order and spelling differences never show as lines.
  • A change at a container's own path with no before (an object set to null, a set emptied) is built from deployed, each leaf from its change.
  • Otherwise a position a change names takes that change's before, which is masked across both sides, so a variable locked since the deploy never shows its old value.
  • A set (a digest segment under a list) is taken from deployed in the current order; intent is taken whole (the two vocabularies spell it differently); objects and lists removed in Checkly are restored from deployed and render as diff lines.
  • Changes under deploy-only or relation keys, unresolvable references and values known only by their hash become notes, never guesses.

Input contract for callers. deployed masks every position that is sensitive at the deploy or now as { $masked: 'same' }; other values known only by hash are { $hash }; references covers every resource the changes or deployed refer to by logical id. The function never mutates its inputs and never throws for a rendering failure (falls back to a listing with a reason).

Validated against real data. A fixture captured from a local environment with the backend's own functions (API and browser check: intent edits, headers, a variable locked since the deploy, legacy doubleCheck, a long script, a group reference, removed alert reminders, location and tag changes) is committed and rendered in diff.spec.ts.

Known limits (v1). Relation drift (subscriptions, assignments) is not an input yet. A whole list added in Checkly whose secret elements cannot be marked inline gets a secret changed: note, as in the preview. A text changed in Checkly whose deployed version is known only by its hash is noted as unavailable.

Tests

  • pnpm --filter checkly test (pack, type tests, all vitest): passes.
  • New: codegen/__tests__/diff.spec.ts (incl. the captured fixture fixtures/drift.json), deploy-diff/__tests__/drift.spec.ts (incl. deployedSide cases) (incl. a coverage table and call-site scan of every spelledOut( / spelledOutUnder( in the codegens), construct-diff.spec.ts, a context: Infinity case in diff-lines.spec.ts, and v2/v3 status page cases in render.spec.ts.

🤖 Generated with Claude Code

sorccu and others added 6 commits November 17, 2026 19:42
… renderer [RED-1547]

The deploy preview's construct diff (render both sides with the import
codegen, mark moved secrets with sentinels, diff the texts, add the content
diffs and notes) moves into construct-diff.ts with explicit before and after
sides, labels, render settings and an origin note, so another caller can
render a diff between two import-format sides. The preview keeps its output:
render.ts supplies its labels, its project registration, the loaded project's
defaults and its CLI-upgrade note. import-shape.ts exports substitute() and
the reference key tables for the same reuse.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…odegen [RED-1547]

generateConstructDiff takes a resource as Checkly holds it now, in the import
format, and the changes the deploy diff reports for it since the last deploy,
rebuilds the deployed side from them and renders the two as a construct diff
with the import codegen, deployed first. Properties that differ are spelled
out on both sides, so a change to or from a default reads as a changed line;
secrets are masked on both sides; a change it cannot place (an object set, a
key the import format lacks, an unknown reference, a hashed value whose
deployed text is gone) becomes a note rather than a guess. Rendering failures
fall back to a listing, the entry stays free of side effects, and the deploy
preview is unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…-1547]

The backend reports an object removed since the deploy by its leaves alone,
with no change at the object's own path, so the rebuild of the deployed side
had no container to create it under and noted each leaf instead. It now
recreates an absent node under an object the payload holds when every change
under it restores a leaf that decodes, none passes through a list index, and
none is a set element, recognised by its key being a prefix of the digest the
backend keys set elements by.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…1547]

generateConstructDiff now takes the resource as its last deploy left it
(`deployed`, in the deploy payload's vocabulary) instead of rebuilding it
by undoing each reported change on the current resource. Positions no
change reaches equal the current resource; a reported value comes from
its change's masked `before`; sets, objects and lists removed in
Checkly are taken from the deployed resource, so they render as diff
lines instead of notes. The undo heuristics are gone.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ults are given [RED-1547]

generateConstructDiff defaulted projectDefaults to {}, a project without
defaults, so a property at its default value was left out. With no
defaults given, it now writes every property, as generateConstructFiles
does for the same option, so the diff reads like the code generated for
the resource. Passing {} keeps the previous output.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
generateConstructDiff masked every value a redaction rule reaches, so an
empty password rendered as '********' with a basicAuth block the
resource does not have. In the drift diff an unconditional rule (a
password, a header value) now leaves an empty value ('', null or absent)
as it is. A conditional rule (a locked or secret variable) still masks
whatever the value, since such a variable read without decrypting holds
null. The deploy preview and write-back keep masking empty values: there
the API blanks credentials to '', so an empty value cannot be told from
a blanked one.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant