Repository navigation
SK-3295 Flowvault release/26.11.1 - #355
Open
skyflow-bharti wants to merge 13 commits into
Open
skyflow-bharti wants to merge 13 commits into
skyflow-bharti wants to merge 13 commits into
Conversation
…Node SDK (#348) * SK-3225: Split SDK into common/skyvault/flowvault; add FlowVault Node SDK Restructure the single `skyflow-node` package into an npm-workspaces monorepo mirroring the Java SDK split (common / skyvault / flowvault). - common (private, bundled into each published package): the contract-agnostic core — errors, utils, jwt, logging, service-account auth, config/credentials, generic validations, mutable SDK identity, Base* request/response/options classes, the BaseSkyflow facade base, and the generated auth client. Builds and tests in isolation. - skyvault -> skyflow-node (2.x): the existing contract, public API unchanged (74 exports, byte-for-byte). Its Skyflow/models extend the common Base* classes; keeps connections, detect and its data-plane generated client. - flowvault -> skyflow-flowvault-node (1.x): new unary v2 contract (insert/get/update/delete/query/detokenize/getTokens/uploadFiles/deleteFiles) wired to the generated v2 client; reconciled with the Java/Python flowvault SDKs. Each layer re-exports common and bundles it (tsup) into a self-contained tarball (verified via npm pack); the mutable SDK identity makes each product self-report its own name@version. Independent versioning: plain-semver tags publish skyflow-node, flowvault/* tags publish skyflow-flowvault-node. Tests: common 232, skyvault 878 (backward compatibility preserved), flowvault 132 (~99.85% coverage). Per-package api-extractor contract snapshots gate the public API of both SDKs. PR CI builds and tests both SDKs with one command. Docs: docs/sdk-split-plan.md. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * SK-3225: fix PR CI (eslint ignores + gitleaks allowlist for generated/dist) - eslint.config.mjs: ignore all generated dirs (**/_generated_/**, legacy leading-space ** / _generated_/**), all workspace test dirs (**/test/**), and build output across workspaces — the old ignores only matched the single-package paths. - Replace three banned magic-string comparisons in hand-written code (typeof === "symbol"/"function") with equivalent non-literal checks. - Rule/gitleaks.toml: allowlist the maintainer-owned generated dirs (example JWTs) and the bundled dist/ output. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * SK-3225: fix skyvault jest — don't babel-transform compiled common/lib CI resolves the skyflow-common symlink to its realpath (outside node_modules), so babel-jest was transforming the compiled common/lib and injecting core-js (an undeclared dep) -> suites failed to run. Add `common/lib` to transformIgnorePatterns so the already-CJS common package is loaded as-is. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * SK-3225: add flowvault branch-coverage tests to clear codecov/project Covers the defensive edge branches the codecov/project gate flagged (net-new flowvault code dipped overall coverage 0.44% vs base): - empty/absent `records`/`response` bodies map to [] across all ops - per-record errors with no x-request-id header -> requestId null - getRequestId with an absent raw response - uploadFiles: non-Error PUT rejection, no-name/trailing-dot content-type, response record with no matching prepared columns - VaultClient: default log level, unrecognized cached auth type flowvault branches 92.51% -> 96.89% (statements/lines 100%); 132 -> 153 tests. No source/generated/config changes. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * SK-3225: flowvault insert/update — per-record table + separate tokens/data Aligns the Node flowvault insert/update contract with the Java/Python SDKs: - InsertRecord/UpdateRecord gain an optional per-record `table`; when any record sets it, a single insert/update call targets multiple tables (the generated InsertRecordData/UpdateRecordData already carry tableName). - Insert/Update response records now expose `tokens` and `data` separately instead of a single merged `fields`. Updated unit tests (incl. multi-table coverage) and the api-report snapshot. flowvault 156 tests pass (100% lines, branches 96.97%); common 232, skyvault 878 unchanged. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * SK-3225: add flowvault README + samples (parity with python flowvault) Expand flowvault/README.md into a full SDK guide (overview, install, auth, client init, all 9 operations, custom headers, error handling) and add flowvault/samples/ — one runnable TypeScript example per operation under vault-api/, plus service-account token-generation samples — mirroring the skyflow-flowvault-python samples. All samples type-check in strict mode against the built SDK types. flowvault-only; skyvault and common untouched. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * SK-3225: flowvault timeout/retry config + wire custom request headers Config (parity with python flowvault, limited to what the Node generated client can honor): - VaultConfig gains optional timeout (seconds), maxRetries, and vaultUrl; the flowvault VaultClient forwards timeout/maxRetries as the generated request's timeoutInSeconds/maxRetries, and vaultUrl overrides the derived URL. (connect/read/write timeouts and retry-delay knobs are fixed in generated code.) Custom headers: - options.setRequestHeaders(...) is now actually wired — merged into every operation's request headers. Added the missing options params to delete, query, getTokens, uploadFiles, deleteFiles so headers work on all 9 ops. Samples/README updated; added tests (162 pass, 100% stmts/lines). flowvault + the shared VaultConfig only; skyvault untouched; api-report unchanged. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * SK-3225: flowvault parity — per-record insert upsert + multi-table get fields Closes two parity gaps vs the Java/Python flowvault SDKs (generated client already supports both; hand-written layer only): - InsertRecord gains optional per-record `upsert`; when any record sets it, each record sends its own upsert (falling back to the request-level one) and the request-level upsert is dropped. - GetRecord (multi-table get) gains optional `columns` / `columnRedactions` / `uniqueValues`, wired per record. Added tests (164 pass, 100% lines); api-report regenerated. flowvault-only; skyvault and common untouched. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * SK-3225: flowvault update updateType + align README/samples with python - UpdateRequest gains an optional request-level `updateType` ("UPDATE" | "REPLACE"), wired to the generated UpdateRequest.updateType (no generated change needed). Closes the last parity gap vs Java/Python flowvault. - README + samples updated to match the python flowvault SDK: per-record `upsert` on insert, multi-table get per-record columns/redactions/uniqueValues, and update `updateType`. 166 tests pass (100% lines); api-report regenerated; samples type-check. flowvault-only; skyvault/common untouched. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * SK-3225: fix flowvault vault URL — use skyvault.* host, not vault.* FlowVault (v3) is served from {clusterId}.skyvault.skyflowapis.{dev|com| -preview.com|tech}, a different subdomain than the shared getVaultURL (vault.*, used by skyvault/v2). The facade was reusing getVaultURL, so every call 404'd. Added a flowvault-local getFlowVaultURL with the four confirmed skyvault.* hosts and used it in buildVaultEntry. flowvault-only; skyvault/common untouched. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * SK-3225: fix flowvault insert/update table placement + per-record error extraction Two bugs surfaced running the e2e against the live vault: - Insert/update sent tableName at BOTH the request level and per-record when any record carried its own table, which the API rejects (400 "table name ... at one place"). Now the request-level tableName is omitted whenever per-record tables are used (table lives in exactly one place), matching Java/Python. - A per-record failure returned with a 4xx HTTP status (e.g. delete/get/update of a non-existent id -> 404 whose body carries a records/response array) was thrown as a whole-call error. Now those per-record records are surfaced as a normal response (via unaryError), and genuine whole-call errors still throw. Added error-extraction tests for insert/get/update/delete/detokenize/getTokens/ deleteFiles. 174 tests pass (99.4% lines). flowvault-only; skyvault/common untouched. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * SK-3225: flowvault get — allow empty ids when uniqueValues are set validateGetRequest rejected empty ids even for a get-by-unique-value, where the lookup is by GetOptions.setUniqueValues (not ids). It now accepts a single-table get with empty ids as long as uniqueValues are provided on the options ("ids OR uniqueValues", matching Java/Python flowvault). 175 tests pass. flowvault-only; skyvault/common untouched; api-report unchanged. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * SK-3225: flowvault custom headers — typed interceptor + CustomHeaderKey Replace the arbitrary setRequestHeaders(Record<string,string>) on options with the same typed mechanism the Python and Java FlowVault SDKs expose: a RequestInterceptor set via options.setInterceptor(), which receives a RequestContext and may attach only the supported headers in CustomHeaderKey (x-skyflow-account-id, x-skyflow-account-name, x-request-id). The context carries the operation name (INSERT, GET, ...) so headers can vary per call. - New: CustomHeaderKey enum, RequestContext (+ RequestInterceptor type), FlowVaultOptions base carrying the interceptor. - All 9 *Options now extend FlowVaultOptions; controller.requestOptions runs the interceptor per operation and maps CustomHeaderKey -> wire header. - README, sample, and tests updated; contract snapshot regenerated. common and skyvault are untouched. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * SK-3225: flowvault docs — vaultUrl override uses the skyvault.* host The vaultUrl config overrides the URL the SDK actually calls, and FlowVault is served from the skyvault.* subdomain (getFlowVaultURL), not vault.*. Fix the two override examples so a user who copies them points at the right host. The clusterId derivation comments keep vault.skyflowapis.com to match the Python SDK (that is where the clusterId is read from).
The internal (JFrog) build's 'Commit Changes' step force-pushed the version bump to the release branch, which failed with a permission denial (the PAT_ACTIONS owner lacks push access) and killed the job before Publish. The push-back is also unnecessary — Publish uses the already-bumped working tree — and harmful, since src/version.ts is not path-ignored so the push would re-trigger this workflow in a loop. Restrict the version-bump commit to public/beta releases; internal builds now bump, build, and publish to JFrog without mutating the branch. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…nfig npm config set cannot run from inside a workspace package directory: npm detects the implicit workspace and aborts with ENOWORKSPACES, which killed the internal Publish step. Write the JFrog registry + auth to a local .npmrc in the package dir instead (npm publish itself supports workspaces and runs fine). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
A workspace member's own .npmrc is ignored by npm (it read the root config and defaulted to registry.npmjs.org, so publish 404'd on public npm). Pass the JFrog registry with --registry on the publish command (highest precedence) and put its auth in the workspace-root .npmrc that npm actually reads. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
samples-local/ holds real creds in .env and is for manual local testing; it must never be committed. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- Add a typed Token { token?, tokenGroupName?, path? } and surface response
tokens as Record<column, Token[]> on insert/get/update, parsing the wire
token map (list / bare value / multiple groups) the same way the Java and
Python flowvault SDKs do. Export Token; regenerate the api-report snapshot.
- Normalize the query data plane's system key skyflowID -> skyflowId in rows
and metadata.columns, so query matches every other operation's response.
- Update unit tests and the README accordingly.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…ntegration-in-flowdb-and-skyvault-sdks SK-3076 added detect support
|
✅ Gitleaks Findings: No secrets detected. Safe to proceed! |
|
Semgrep Findings: Issues with Error level severity are found (Error is Highest severity in Semgrep), Please resolve the issues before merging. |
Public API surface changesCompared with skyflow-node: 728 added, 381 removed (base report read from its old location
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Detect support and vault api in flowvault sdk