Skip to content

feat(task-steps): prompt steps store each turn's atif/1 copy when the run prescribes it - #854

Open
earakely-scale wants to merge 3 commits into
mainfrom
edgararakelyan/fd-6078-y4-atif-copies
Open

earakely-scale wants to merge 3 commits into
mainfrom
edgararakelyan/fd-6078-y4-atif-copies

Conversation

@earakely-scale

@earakely-scale earakely-scale commented Nov 20, 2026 •

Copy link
Copy Markdown
Collaborator

Y4 PR 2 of 3 for FD-6078. When the run prescribes atif/1 (Y16, #849), a prompt step stores each solver turn's atif/1 copy beside its native one, with its aux log, turn context and media, checks it, and records how it checked. Judges keep reading native. Refs FD-6078, which stays open for PR 3 (always_on and the live follow).

What changes

  • Each turn's copy (task_step/snapshot_utils/atif_copies.py, AtifCopies). Once a turn ends, a prompt step whose run prescribes atif/1 reads that task's atif/1 document, and it does so only then.
    • The read is inline ({task_id, format: "atif/1"}), with the 600 s transfer timeout (Y4-8). The answer carries the document, the aux log and the turn context.
    • The worker stores the document as <trajectory_id>.json in the run's folder for the agent, {prefix}atif/instance_id=<id>/<agent>/, and overwrites it each turn (Y4-2). That folder is recorded in metadata["atif_copies"]. A run with no instance id gets one adhoc-<uuid12> for all its agents (Y3-2).
    • The aux log and turn context go to turns/<message id>/aux.json and turn_context.json, named by the id the native copy is stored under.
  • Off the turn's path. Each turn's copy is a background task, chained after the previous turn's so overwrites land in turn order. The step waits for all of them only before it appends its prompt response. The user side's next message never waits for a copy. A step that fails cancels the copies still running.
  • Media (Y3-1, Y3-5, Y3-7).
    • The worker lists the folder's media/ once per step.
    • Each turn, it asks the agent's media push for the files the document names that the folder lacks. Each file gets an exact-object grant sized from the profile, through the staging store when the store's grants don't reach the agent.
    • A file the agent answers missing or failed for, or whose push answer was lost, is looked up in the store before it counts as missing. A repeat pull asks only for what the store lacks.
    • A file larger than the store's largest single upload isn't asked for.
    • Each held file's size comes from the push's answer, or once per step from the store's metadata. media-bytes also flags a profile whose sha256 isn't the file's name, or whose size isn't the stored file's.
  • Staged calls (a2a_agent/object_transfer.py, invoke_transfer). A staged call that fails after it's sent, its answer lost or refused, still pulls what the agent wrote before the staging is released. An upload through a grant already reaches the store whether or not its answer arrives. pull skips what the agent didn't write, and a failure there is logged so the call's own error is the one raised. Without this, a media push whose answer was lost on the staging route would lose files the agent had already let go of.
  • The turn's record (Y4-1, Y4-3, AtifRecord in task_step/context.py). Each turn's target_agent_per_turn_trajectories entry now fills atif with {url, through_step, status, rules, reason, validator_version, adapter_version, aux_url, turn_context_url}.
    • valid: the producer's check, plus media, found nothing.
    • invalid: the copy is kept, rules lists the rule ids it broke, and reason gives the first. A media file still missing after the pull counts under media-bytes. A document whose shape can't be walked (say steps: [null]) is stored too, recorded with the validator's rules, its media and session left alone.
    • newer: the schema or profile version is newer than the worker's, so it's checked only as a reader would, with those rules.
    • unavailable: reason says why. It's the agent's error code when it gives one (format_not_prescribed for a conversation started unprescribed, per PF-4). Otherwise it's http_<status>, timeout, unreachable, no_document, invalid_trajectory_id, or the error's class.
    • atif is None only when the run doesn't prescribe atif/1. Prescribed to an agent whose card doesn't offer it, a turn's record is unavailable with format_not_offered (Y4-7), so one turn's record shows whether the run asked and whether it got it.
  • Continuation links (Decision 18). When a conversation's next copy is a new document of the same session, as after a restart, the worker sets continued_trajectory_ref on the session's earlier stored document to <new id>.json. The agent that wrote the earlier one may be gone. The session names the new document only once that's done, so a link that fails is made by a later turn.
  • Never fails the run. Any error in a turn's copy becomes that turn's unavailable record, with a warning.
  • Docs. AGENTS.md's [trajectories] bullet and the config template's comment cover the stored copies.

Choices to review

  • Inline, not uploaded (Y4-8, decided with Edgar). The worker can't know trajectory_id before it reads, and a grant to a guessed name could overwrite the earlier document a restart leaves. Native reads keep the objects form.
  • A trajectory id that isn't a plain file name is refused. The agent mints the id and the worker writes a store key from it, so the id must match [A-Za-z0-9][A-Za-z0-9._-]{0,127}. Anything else is unavailable, invalid_trajectory_id, and nothing is written.
  • Held media aren't read back. The SDK checks each file's sha256 against its name before the bytes leave and after the upload (feat(a2a-agent): an atif/1 copy's media files wait on the agent's disk until the worker pulls them #834). So the worker's media-bytes check is that the folder holds each file, not that it re-hashes them. Stored copies still validate with --media, as the tests check.
  • A pinned image in a prescribed run. A card with no formats gets today's native request (Y4-6), and its atif record is unavailable, format_not_offered (Y4-7). In a run that doesn't prescribe, it's None, as before.
  • A restart within a step. The prescription is agent-wide and held in memory (PF-2), so an agent that restarts partway through a step builds native only for the rest of that step. Its turns are recorded unavailable, format_not_prescribed. The next step prescribes again, and its copies link from the earlier document. Keeping the prescription across a restart, as the turn counts are kept, would be an SDK change for later.
  • A staged call that fails still collects its objects. This changes invoke_transfer for every staged call, not only the media push. A failed snapshot save or trajectory get on the staging route now leaves in the store what the agent had written, as the same call through a grant already does.
  • No records for an agent without the trajectory extension, in a prescribed run as in any run. The records line up with the native url list, which judges read and which stays empty for such an agent. The step's metadata["trajectory_formats"] entry (prescribed beside built: None) and its warning say the run asked and the agent can't build it.
  • validator_version is the installed agentenv-framework-protocol version. adapter_version is the copy's own extra.agentenv.adapter.version.
  • always_on, user sims and the live follow don't store copies yet: PR 3 does always_on and the live follow, and Y15 (FD-6441) does user sims.

Testing

  • Unit tests, new in tst/unit/task_step/test_prompt_agent_atif_copies.py: 20 tests. A prompt step runs against an in-process SDK agent whose adapter puts a screenshot in each turn, and whose uploads land in a granting local store. They cover:

    • two turns stored as one document, with aux log, turn context and media, validating with the folder's media;
    • nothing read or stored without a prescription;
    • format_not_offered;
    • format_not_prescribed after a failed prescription;
    • invalid with its rules;
    • newer, by schema and by profile version;
    • media-bytes when the card lists no media push;
    • a lost push answer, found in the store and not asked for again in that step or the next;
    • the restart link between two steps;
    • the user side asked while the turn's atif/1 read is still held;
    • four unsafe trajectory ids;
    • the folder of a run with no instance id;
    • a link whose store read fails once, made by the next turn, which keeps its copy;
    • a document that doesn't read as ATIF, stored and recorded invalid;
    • a profile that misdescribes a stored file's bytes or sha256.

    And in staging_test.py: a staged trajectory get whose answer is lost still leaves the agent's object in the store, in both staging variants. Each of the last four fails on the code before the fix.

  • Planted bugs. Each of these fails at least one test:

    • the read awaited on the turn's path;
    • the store not consulted after a lost answer;
    • no continuation link;
    • None instead of format_not_offered;
    • no id check;
    • no media check.
  • The judge-input pin. Its "with records" form now also stores an atif/1 copy in the run's folder, fills each turn's atif record and records atif_copies. All 18 cases still match the golden file.

  • Existing tests. The prescribe test now expects the step's own atif/1 read. Locally the related unit folders pass (task_step, a2a_agent, openenv, task, config: 5,280 tests). The 4 failures are the known Mac-only computer_exec_test.py ones. ruff check . passes.

  • Integration. test_prompt_agent_live_trajectory_local.py's format test now sends the deployed echo agent an image with its prompt. It checks that the turn's record is valid, that the stored document is the one served, that the turn context is stored, and that the image's file sits in the folder and validates. No new test, so CI time doesn't grow. On an amd64 test host the file's 5 tests pass, at 80fc681 (73 s) and again at e76cb71 (61 s).

  • Chaos (chaos_copies.py): real prompt steps against an in-process agent.

    • Each seed runs 1-3 steps of 1-3 turns. The agent builds atif/1 or native only, with or without the media push, and is sometimes restarted between steps.
    • Faults: the prescription unreachable; a read answered 500, timed out, not a document, with an unsafe id, newer, with a stray key, with a step that isn't one, or with a profile that misdescribes its media; a media push down, or its answer lost.
    • A model tracks what the agent was last prescribed and what each conversation was built with. It checks each turn's status and reason, that a valid copy is the one served and validates with its media, that each stored document is the last one served (linked after a restart), and that nothing is written outside the two prefixes.
    • Locally, 40 s ran 1,194 seeds clean. Three planted bugs were each caught within seconds: no store check, no link, and copies stored out of order. It also caught a first try at Greptile's malformed-copy finding that gated the media walk on the validator, which skipped newer copies' media and links.
    • On the test host: 20 minutes at 80fc681 ran 43,478 seeds clean: 86,978 steps and 173,949 turns, with 23,337 valid, 12,995 invalid, 5,252 newer and 46,439 unavailable records, 13,116 restarts and 1,664 links checked. After the review fixes, with the two new faults, 15 minutes at e76cb71 ran 32,617 seeds clean: 65,110 steps and 130,347 turns, with 14,976 valid, 14,758 invalid, 3,210 newer and 32,164 unavailable records, 9,833 restarts and 1,186 links checked.

🤖 Generated with Claude Code

RetriggerView in GreptileConfidence Score: 5/5

The PR appears safe to merge, with the earlier non-blocking gap for agents without a trajectory extension still present.

Summary

Prescribed prompt steps save each turn's atif/1 copy, media, and check results beside the native trajectory.

  • Prompt turns store a checked atif/1 copy when the run prescribes it.
  • Stored atif/1 copies include and check the media their documents name.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart LR
  A[Solver turn ends] --> B[Store native trajectory]
  B --> C[Ask user for next message]
  B --> D[Queue ATIF copy]
  D --> E[Read document and collect media]
  E --> F[Check and store turn record]
  F --> G[Wait before saving prompt response]
Loading

Reviews (4) · Last reviewed commit: "fix(a2a-agent): what an agent staged bef..." · Reviewed by Greptile

… run prescribes it

A run that prescribes atif/1 reads each solver turn's copy inline once the turn ends, off the turn's
path, and stores it as <trajectory_id>.json in the run's folder for the agent, with its aux log, turn
context and media; each turn's record says how it checked (valid, invalid, newer, unavailable), and a
restart's next document is linked from the session's earlier one. Refs FD-6078.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@earakely-scale
earakely-scale marked this pull request as ready for review November 20, 2026 23:03
@earakely-scale
earakely-scale requested a review from a team as a code owner November 20, 2026 23:03
earakely-scale and others added 2 commits November 20, 2026 15:27
… invalid, and media are checked against the profile

The session names a new document only once the earlier one links to it, so a later turn retries a link
that failed. A copy whose shape can't be walked is stored and recorded invalid by the validator's
findings, rather than unavailable. A held media file whose profile sha256 or size disagrees with its name
or the stored file is media-bytes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…he store

A staged call whose answer is lost or refused still moves the objects the agent wrote into the store, as
an upload through a grant does, so the atif/1 media pull finds a file the agent has already let go of.
Names the atif/1 record's reason limit.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant