Skip to content

feat(sdk): add an account keystore and recovery-phrase accounts - #90

Merged
iamalwaysuncomfortable merged 6 commits into
masterfrom
feat/account-keystore
Nov 17, 2026
Merged

iamalwaysuncomfortable merged 6 commits into
masterfrom
feat/account-keystore

Conversation

@iamalwaysuncomfortable

@iamalwaysuncomfortable iamalwaysuncomfortable commented Nov 17, 2026 •

Copy link
Copy Markdown
Member

Summary

Adds aleo.keystore, a small account store for the Aleo SDK, and recovery-phrase support on the account module. A bot or script can store an account once and resolve it on every run without carrying its own key-handling code.

The account file format is shared with the TypeScript SDK (Veil), so an account stored by either SDK opens in the other. The companion change there, ProvableHQ/veil#221, adds the same keystore to @provablehq/veil-aleo-sdk/node.

What is added

aleo.keystore

Function Effect
save(account, …) Stores an account in ~/.aleo/accounts/<name>.json. Never replaces an existing account file.
load(…) Resolves the key for this process. Never generates or stores one.
info(…) Describes an account file without reading its key.
protect(…) Changes how an account holds its key, or sets a new password.
export(…) Returns the stored key.
migrate(source, …) Copies a key out of an older JSON state file.
  • Storage types: plaintext (the default; an owner-only file), encrypted (a password stretched with scrypt, then the SDK's own private-key ciphertext), keychain (the macOS login keychain) and none (the address only).
  • Lookup order: an explicit key, ALEO_PRIVATE_KEY, ALEO_PRIVATE_KEY_FILE, the account file, then a caller-supplied legacy key. SHIELD_SWAP_PRIVATE_KEY and SHIELD_SWAP_PRIVATE_KEY_FILE are accepted as aliases.
  • Errors: one KeystoreError (an AleoError) with a stable code.

python -m aleo.keystore with create, import, show, protect, export and migrate.

Recovery phrases on the client's account module: from_mnemonic, generate_mnemonic and create_with_mnemonic, using the derivation of the Shield wallet and the TypeScript SDK. The BIP-39 English word list ships as package data.

Behaviour worth reviewing

  • A key supplied through the environment is used for the process and never written to disk.
  • An environment key must match the selected account file's address; a mismatch is an error naming both addresses.
  • A variable that is set but empty is an error, and two variables that supply different keys are an error.
  • An unknown file version or storage type stops with an error. It never falls back to another source.
  • Passwords are never taken from the command line or written to a file. They come from the password argument, ALEO_ACCOUNT_PASSWORD, a password saved in the macOS Keychain, or a hidden terminal prompt.
  • Commands that print a key or a recovery phrase (export, create --storage none, create --mnemonic) run only with a terminal attached and refuse --json.
  • A new account file is created exclusively, so two processes storing different accounts under one name cannot replace each other's file.
  • protect reads the new storage back before removing the old secret, and restores the original file if any later step fails.
  • protect holds an interprocess lock (<name>.json.lock) while it changes an account file, and stops with ACCOUNT_BUSY if another process holds the lock or changed the file in the meantime. A lock is never removed automatically; the error names the file.
  • Each account file has a Keychain key entry of its own, named after the address and a random identifier that the file records in key.account. Files stored separately never share an entry, so creating, undoing or removing one file's entry cannot affect another's. The label carries the account name (aleo-account-key:<name>) for display only.
  • A saved password belongs to one encrypted file: its entry is named after the address and the file's salt, so two files that hold one account under different passwords do not share it.
  • Encryption and the Keychain protect a key at rest. They do not stop a process running as the same user from reading it; the README says so.

No existing behaviour changes. aleo.account.create(), from_private_key(), export_encrypted() and import_encrypted() are untouched, and no runtime dependency is added.

Not in this change

  • aleo_shield_swap (Profile.load_or_create()) still stores its key in profile.json. Moving it onto the keystore is a follow-up.

Testing

  • python -m pytest python/tests -m "not slow": 1503 passed, 14 skipped.
  • pyright: 0 errors.
  • New tests: test_keystore.py, test_keystore_cli.py, test_facade_mnemonic.py. They run in temporary directories with the key, account and password variables cleared and the Keychain tool replaced by a fake.
  • One opt-in test uses the real macOS Keychain (ALEO_KEYSTORE_KEYCHAIN_TEST=1), with the public sample account only, and removes its entries afterwards. Run locally on macOS: passed, nothing left behind.
  • Cross-checked against the TypeScript implementation: each side opened the other's plaintext, address-only and encrypted files; plaintext and address-only files are byte-identical; a set of malformed account files produced the same error code in both; and 40 freshly generated recovery phrases in each direction derived the same addresses.
  • CI runs the suite on Linux, macOS and Windows. The Keychain paths are exercised there through the fake; the real Keychain test runs only on macOS when opted in.
  • Not exercised: the Keychain when locked or over a remote login.

Sample files

sdk/python/tests/fixtures/keystore-vectors/ holds the shared sample files. The private key in them is the devnode key that @provablehq/veil-aleo-devnode already publishes, and the recovery phrase is the standard BIP-39 test phrase. Neither is a new secret, and neither account should ever be funded.

Add aleo.keystore, which stores an Aleo account in
~/.aleo/accounts/<name>.json and resolves it again on later runs:

- save, load, info, protect, export and migrate, with a KeystoreError
  carrying a stable code.
- Four storage types: plaintext (the default, an owner-only file),
  encrypted (a password stretched with scrypt), keychain (the macOS
  login keychain) and none (the address only).
- One lookup order: an explicit key, ALEO_PRIVATE_KEY,
  ALEO_PRIVATE_KEY_FILE (with the SHIELD_SWAP_ names as aliases), the
  account file, then a caller-supplied legacy key. A key from the
  environment is never written to disk, and lookup never generates one.
- python -m aleo.keystore with create, import, show, protect, export
  and migrate. Commands that print a key or a recovery phrase run only
  at a terminal.

Add recovery-phrase support to the account module: from_mnemonic,
generate_mnemonic and create_with_mnemonic, using the derivation of the
Shield wallet and the TypeScript SDK. The BIP-39 English word list ships
as package data.

The account file format and the sample files under
tests/fixtures/keystore-vectors are shared byte for byte with the
TypeScript SDK, so either SDK opens accounts the other stored.
- Create a new account file exclusively, through a hard link, so two
  processes storing different accounts under one name cannot replace
  each other's file. A save that loses the race removes the Keychain
  entries it wrote.
- Remove Keychain entries written for an account file that could not be
  created.
- protect: when a step after the new file is written fails, restore the
  original file and remove a Keychain key entry written for the new
  storage. Bring the saved password in line before the old Keychain key
  is removed, so a failure there leaves the original account working.
- migrate: remove both key fields when a legacy file holds the same key
  in each, and refuse with KEY_CONFLICT when they hold different keys.
- Skip the owner-only re-mode of an account file where os.fchmod does
  not exist. Windows has no fchmod and does not enforce these modes.
- Keep the shared sample files and the BIP-39 word list at LF line
  endings on every platform, since they are checked byte for byte.
- Drop a test that asserted the package under test was imported from
  the source tree, which does not hold for an installed package.

@iamalwaysuncomfortable iamalwaysuncomfortable left a comment

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Found two issues that should be fixed before merging: shared Keychain entries can be deleted by another account file, and saved passwords collide between encrypted files for the same address.

Validation: the focused keystore, CLI, and mnemonic suite passed (353 passed, 1 skipped). Both issues were reproduced with a simulated Keychain; the real Keychain was untouched.

Comment thread sdk/python/aleo/keystore/__init__.py Outdated
Comment on lines +428 to +431
def remove_written() -> None:
for service in written:
try:
delete_keychain(service, account.address)

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P1] Preserve shared Keychain entries during cleanup

Keychain entries are shared by address, but this cleanup assumes the failed save owns the entry. Reproduced: save an account as original with Keychain storage, then save it under another name while simulating a disk-write failure. The cleanup deletes the existing key, and loading original now fails with KEYCHAIN_ITEM_NOT_FOUND. This can erase the only persisted copy. The deletion in protect() also breaks other files sharing the entry. Preserve pre-existing/shared entries when rolling back or changing storage.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 7642b77. write_keychain now reads the entry first and reports whether the call created it, and rollback removes only entries the call created. An entry that already holds the key is not rewritten.

For protect(), the old key entry is kept when another account file in the same directory still points at it. Account files in other directories are not visible to the keystore, so a file elsewhere that shares the entry is still affected when the last local file leaves Keychain storage.

The same commit also puts the removal of the old secret inside the rollback, matching the fix in ProvableHQ/veil#221.

Comment thread sdk/python/aleo/keystore/__init__.py Outdated
Comment on lines +330 to +334
key = dict(encrypt_key(private_key, password))
if save_password and keychain_available():
write_keychain(PASSWORD_SERVICE, address, password_to_keychain_value(password), location.name)
if written is not None:
written.append(PASSWORD_SERVICE)

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P2] Scope saved passwords to the encrypted file

The saved-password entry is keyed only by address, although separate files for that address can use different passwords. Reproduced: save the same account under two names with different passwords and save_password=True; both saves succeed, but loading the first now raises WRONG_PASSWORD because the second overwrote its saved password. That failure also prevents fallback to a terminal prompt. Associate saved passwords with the encrypted file rather than the address alone.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 7642b77 with a file-specific identity. The saved-password entry is now named <address>-<salt in hex>, where the salt is the encrypted file's own scrypt salt. Two files for one address have separate saved passwords, in any directory, and a load never consults another file's entry, so the first file's unattended load keeps working and a file with no saved password still reaches the terminal prompt. The account file format is unchanged.

Tests: test_gives_each_encrypted_file_its_own_saved_password, test_does_not_consult_the_saved_password_of_another_file_for_the_same_address.

A Keychain key entry is found by address, so every account file that
holds one key in the Keychain points at the same entry. The keystore
treated an entry as owned by the file being written, and could remove a
key another file relied on.

- Report whether a Keychain write created its entry, and undo only the
  entries a call created. A save that fails after finding an existing
  key entry leaves it in place, and an entry that already holds the key
  is not rewritten.
- Give each encrypted file its own saved password. The entry is named
  after the address and the file's salt, so two files that hold one
  account under different passwords no longer overwrite each other's
  saved password, and a load never consults another file's.
- protect: undo every step together. The new file, its saved password
  and the removal of the old secret now share one rollback, so a failed
  rotation keeps the original saved password and a failed removal of
  the old key restores the original storage.
- Keep a Keychain entry that a neighbouring account file in the same
  directory still uses when another file leaves that storage.

@iamalwaysuncomfortable iamalwaysuncomfortable left a comment

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-reviewed 7642b77. The saved-password collision is fixed, and a failed save now preserves a Keychain entry that existed before the call. Shared Keychain deletion still has two failure cases: an overlapping writer can adopt a newly created entry before rollback, and account files in other directories are omitted from the ownership check. Both findings should be addressed before merging.

Validation: the focused keystore, CLI, and mnemonic suite passed (362 passed, 1 skipped). Both remaining failures were reproduced with a simulated Keychain; the real Keychain was untouched.

Comment on lines +513 to +515
except BaseException:
# No account file was created, so nothing points at the entries made here.
_remove_entries(created)

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P1] Preserve entries adopted by another writer before rollback

Creating an entry does not guarantee exclusive ownership until this call finishes. Reproduced this interleaving: writer A creates the Keychain entry; writer B successfully saves the same key under another name; A's file write then fails. This rollback deletes the entry B now relies on, leaving B's successful save unable to load (KEYCHAIN_ITEM_NOT_FOUND) and removing the only persisted key. Creation, adoption, and cleanup need coordinated ownership handling.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in dc715a6 by removing the sharing, so there is nothing to adopt. Each account file now gets a Keychain key entry of its own, named <address>-<16 random bytes in hex> and recorded in the file's key.account. Writer A and writer B create different entries; when A fails it removes only the one it made, and B's is untouched.

Test: test_does_not_let_a_failed_save_remove_the_key_of_a_save_that_succeeds_alongside_it runs the second save between the first save's Keychain write and its file write.

Comment thread sdk/python/aleo/keystore/__init__.py Outdated
Comment on lines +853 to +857
if current == "keychain" and storage != "keychain":
if not _neighbour_uses(
location, lambda file: file.get("address") == address and _key_of(file).get("type") == "keychain"
):
delete_keychain(KEY_SERVICE, address)

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P2] Do not infer global ownership from neighbouring files

The Keychain entry is shared across directories, but this check searches only the selected directory. Reproduced: save the same account with Keychain storage in two project directories, then protect the first as plaintext. The operation succeeds but deletes the second project's key entry; its next load fails with KEYCHAIN_ITEM_NOT_FOUND. Copied encrypted files in another directory similarly lose their saved password. Preserve shared entries when their other users cannot be determined, or track ownership across directories.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in dc715a6 the same way. Two account files stored separately no longer point at one entry, in the same directory or in different ones, so protect on one leaves the other's entry alone. The directory scan is no longer used to decide whether a separately stored account still needs an entry. Saved passwords were already per file (named after the file's salt).

Test: test_leaves_the_same_account_in_another_directory_working_when_one_leaves_the_keychain.

One case still shares an entry: a byte-for-byte copy of an account file, because the copy carries the same entry name or the same salt. A copy in the same directory keeps the entry alive. A copy kept in another directory is not seen, so removing the original's storage removes the entry that copy points at. The key is not lost in that case, since the original's new storage is read back before the entry is removed. Tracking copies in arbitrary locations would need a registry outside the account file, which I have not added.

A key entry was named after the address alone, so every account file
that held one key in the Keychain shared a single entry. No check that
looks at one directory can prove that such an entry is safe to remove:
an overlapping save could adopt an entry that a failing save then
removed, and the same account stored in another directory lost its key
when the first left Keychain storage.

- Name each key entry after the address and a random identifier chosen
  when the file is written, and record that name in the account file's
  `account` field. Entries are no longer shared between files that were
  stored separately, so creating, undoing or removing one file's entry
  cannot affect another's. A reader accepts only an entry name that
  starts with the file's own address.
- A failed save removes the entries it made before anything else, so a
  later error cannot leave them behind.
- Only a copy of an account file still shares that file's entries. An
  entry a copy in the same directory uses is kept.

The shared sample file keychain.json now carries an entry name. A
Keychain file written by the keystore matches it apart from the
identifier it chooses.
Comment thread sdk/python/aleo/keystore/__init__.py Outdated
Comment on lines +878 to +880
except BaseException:
write_file_atomic(location.path, existing.data)
_remove_entries(created)

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P1] Serialize storage changes before restoring an old account file

Rollback unconditionally restores the file captured before this operation, even if another process has since changed it successfully. Reproduced: A reads a Keychain-backed account; B changes that same file to plaintext and deletes the old Keychain entry; A's write then fails. A restores the stale Keychain pointer over B's plaintext file, leaving no persisted key and causing KEYCHAIN_ITEM_NOT_FOUND. Protect the entire read/change/cleanup/rollback sequence with an interprocess lock or equivalent concurrency control.

Validated against dc715a6 using a simulated Keychain and a deterministic interleaving with a simulated write failure. The focused keystore, CLI, and mnemonic suite passes (368 passed, 1 skipped); the real Keychain was untouched.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 2910aaf. protect() now holds an interprocess lock (<name>.json.lock, created exclusively next to the account file) for the whole write, read-back, cleanup and rollback sequence. Under the lock it reads the file again and stops with the new code ACCOUNT_BUSY if the bytes no longer match what it read, so in your interleaving A never writes and never restores: B's plaintext file stands.

Passwords are asked for before the lock is taken, so it is held only while files and the Keychain change. A caller that finds the lock held waits up to ten seconds, then fails with ACCOUNT_BUSY.

A lock is never removed automatically, however old. I first had a stale-lock timeout, but judging a lock abandoned and removing it cannot be one step, so two waiters could both remove it and both proceed. A lock left by a killed process stays until a person removes it, and the error names the file.

Tests: test_does_not_restore_a_stale_file_over_a_change_another_process_made reproduces the interleaving (it fails with the check disabled), plus test_waits_for_the_lock_and_gives_up_with_account_busy, test_never_removes_a_lock_on_its_own_however_old_it_is and test_releases_the_lock_when_the_change_fails. The same change is in ProvableHQ/veil#221, and I checked that a lock held by either SDK blocks the other.

protect() reads an account file, writes a new one, removes the old
secret, and puts the old file back if a step fails. Two processes doing
that to one file could interleave: the first reads a Keychain-backed
file, the second moves it to plaintext and removes the Keychain entry,
then the first fails and restores its stale copy, leaving a pointer to
an entry that is gone and no stored key.

- Hold an interprocess lock, <name>.json.lock, created exclusively,
  while the file and the Keychain change. A caller that finds it held
  waits up to ten seconds and then fails with the new code
  ACCOUNT_BUSY. Passwords are asked for before the lock is taken.
- Under the lock, read the file again and stop with ACCOUNT_BUSY if it
  no longer matches what was read, so a change another process made in
  the meantime is never overwritten or undone.
- Never remove a lock on another caller's behalf. Judging a lock to be
  abandoned and removing it cannot be one step, so two waiters could
  both remove it and both proceed. The error names the lock file so a
  person can remove one left by a killed process.
@iamalwaysuncomfortable
iamalwaysuncomfortable merged commit ed36dec into master Nov 17, 2026
20 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant