A prototype for running agentic workflows for MIA on the Janelia cluster, built on Temporal for orchestration, LSF
for compute, and GitHub issues as the user interface. Three workflows so far: autodev turns a labelled GitHub
issue into a PR written by Claude Code, revises it on request, and merges it after human approval; mia-data
turns a dataset in the MIA catalog into an approved reader binding and reviewed crops, with a human approving every
binding; mia-buffers turns a training-buffer spec into staged OME-Zarr pyramids and a service that hands
multi-resolution crops to training jobs. The three meet in buffer-api's registry and its read-only web page.
Setup: docs/autodev-local-setup.md. Running it under a bot identity:
docs/github-app.md. Production layout: docs/deployment.md.
autodev is an issue → PR → review → merge loop. A person with write access adds the autodev:go label to an issue.
autodev runs Claude Code on the cluster, which implements the issue and opens a PR. The reviewer can request changes
(a review, or a /revise <what> comment) and autodev runs another round on the same branch. When a human approves,
autodev squash-merges that exact commit and closes the issue. Anything it can't handle gets the
autodev:needs-human label and a comment explaining why.
Humans stay in control. Only people with write access can trigger, approve or steer it, agent commits touching
.github/ are refused, and branch protection on main is the real merge gate. Details, trust boundaries and
known gaps: docs/autodev-runbook.md. Design rationale:
docs/agentic-dev-infrastructure.md.
mia-data is a dataset → binding → test crops → human approval → production crops loop. Each dataset that
mia-agentic-search has downloaded gets an issue in
mia-workflow-data. On data:go, a Claude agent in a cluster
sandbox matches the dataset to a reader snippet from mia-snippets
and writes a binding; plain harness jobs check it, cut the round's crops, verify them and show them in the issue.
A human approves with data:approve, and those crops are published to shared storage and a galleries branch; or
rejects with data:reject and comments, which go to the agent fixing the snippet (a new snippet's PR, which autodev
writes, is merged only when its crops are approved). Every stage is a label
and a new comment with links to the folders involved. The whole flow, with a diagram and the component for each
step: docs/mia-data-workflow.md. Design: docs/mia-crops-spec.md.
mia-buffers is a buffer spec → plan → staged pyramids → served crops loop, with no agent in it. A spec (which
datasets by catalog fields, patch size and pyramid levels, splits, weighting, how many samples per volume) is filed
as an issue in mia-buffers. On buffer:go the worker plans it
against the catalog at a pinned commit and the registry's prepared datasets, then sources every dataset of the
pool in parallel: the raw files are fetched into a shared raw cache (one writer per record), and a harness job
stages one sparse OME-Zarr 0.6 pyramid per volume, holding only the chunks that cover the planned samples, with
the derivation (source bytes, code commits, plan, voxel size, verification) recorded in the Zarr itself and
checked against the 0.6 schemas before publication. Each volume is registered with buffer-api as it becomes
servable; the round is staged before a human approves it (buffer:approve; buffer:plan adds a gate after the
plan), and the buffer is ready once every weighting group reaches the spec's minimum. A staged artifact is
addressed by its recipe, so a later buffer with the same binding, levels and plan reuses it instead of staging
again. After that the buffer lives as long as the issue: idle when no consumer has drawn for a while,
buffer:keep to hold it, buffer:retire to release what it references; a keeper evicts unreferenced caches under
the configured quotas and is the only code that deletes.
A dataset that is not prepared is out of the pool. A prep:go request issue hands its records to mia-data
(data:go on their dataset issues) and follows them through the registry until each is prepared, so buffering
never asks an agent anything. Design: the training buffers,
prep and buffering and
sparse OME-Zarr 0.6 staging specs under docs/specs/.
buffer-api (mia-buffers serve) is the one sampler and ledger: it picks each crop by the spec's weights and
splits from the planned samples (train with replacement, val and test from frozen lists), records every draw so a
run can be replayed, and keeps the registry: catalog facts at synced commits, each dataset's preparation state,
what is on disk and which buffers use it. Training code uses mia_buffers.client.BufferDataset, a MuViT dataset
that pulls crop descriptors and tensorises them locally with the mia-snippets harness, so no pixel crosses the
service. Mutating routes need the worker token; consumers can only name their split; the page is read-only.
These workflows mostly wait: for a reviewer, for CI, for an approval, for a cluster job, often for days. Each issue gets one long-lived Temporal workflow that holds its state and receives GitHub events as signals. That gives us:
- Survival of restarts. If a worker or host dies mid-run, the workflow resumes where it was. Job submission is idempotent and polling reattaches to the same LSF job, so nothing is run twice.
- Cheap waiting. A workflow waiting on a human costs nothing while idle, and durable timers replace cron-style bookkeeping.
- Retries instead of glue code. Transient GitHub, git and cluster errors are retried by policy; permanent ones stop the round and escalate to a human.
- One object per issue/PR. Workflow IDs make duplicates impossible, and the Temporal UI shows exactly what happened in each run, which is the main debugging surface.
The tradeoff is that workflow code must be deterministic, so changing it can break in-flight runs. Workflows are kept thin (behaviour lives in activities), and a replay test against a recorded history guards the rest. See the runbook's known gaps.
Agents are untrusted code driven by text from issues, so they never run on the host directly.
- The orchestrator (Temporal dev server and worker) runs inside agentic-sandbox, which gives it read-only access to the code and config, read-write access only to its own state and work directories, and a network allowlist (GitHub and Temporal only).
- Each agent round is submitted to LSF as a job through csub (
csub submit --claude). The job runs Claude Code in its own sandbox, bound read-write to just its round directory, with a network allowlist limited to Anthropic and package hosts. It never gets GitHub access or the worker's token. - A csub broker on the host, running with the user's policy (
broker.toml: allowed hosts, allowed roots,allow_claude), is what actually submits the jobs. The sandboxed worker can only ask it. - The worker, not the agent, does all GitHub and git pushes, and pushes only to the deployment's own
autodev/issue-Nbranches.
| path | |
|---|---|
mia_core/ |
what every package shares: the mia.yaml loader, host-local-disk and secret-file checks, the worker-side registry client, untrusted-output helpers |
autodev/ |
autodev: workflows, activities, CLI; GitHub client and csub job runner shared with the others |
mia_data/ |
mia-data: workflows, activities, harness job specs, CLI (pixi run mia exec mia-data ...) |
mia_buffers/ |
mia-buffers: the MiaBuffer and MiaPrep workflows, the planner, stage job specs, the cache keeper, buffer-api with its ledger, registry and page, and the BufferDataset client |
mia_ops/ |
pixi run mia: starts, stops and inspects every service in mia.yaml (Temporal, autodev, snippets, mia-data, mia-buffers, buffer-api), and mia exec <service> <cmd> runs an app command with that service's config |
docs/ |
setup guides, the mia-data workflow overview and the development guide; design notes, the runbook, specs/ and plans/ |
tests/ |
unit tests (pixi run test); no GitHub, LSF or Temporal server needed. pixi run lint runs ruff, pixi run typecheck runs mypy |