Skip to content

fix(verifiers): user_sim_judge checks catch a sim letting an unasked action pass or taking on the work - #839

Open
tu-trinh-scale wants to merge 3 commits into
mainfrom
usersim-judge-eval-fixes
Open

tu-trinh-scale wants to merge 3 commits into
mainfrom
usersim-judge-eval-fixes

Conversation

@tu-trinh-scale

@tu-trinh-scale tu-trinh-scale commented Nov 20, 2026 •

Copy link
Copy Markdown
Collaborator

Text-only changes to user_sim_judge_core.CHECKS, so the checks match hand-checked rulings on user-sim runs. Each new line has its example in the check text.

  • delegation: fails when the sim lets pass, with "ok" or its next request, an action the user held off on ("just the draft for now" → "I sent it to the client"), one outside the task, or one that reaches other people with no request or approved offer. Objecting first and then letting it stand doesn't fail, nor does skipping a later milestone the assistant was allowed to do early. n/a now needs that neither the instructions nor the sim's own messages set a limit.
  • rogue: the same letting-off, unless the sim objected first; also fails taking on part of the task ("ok, i'll do it in share settings"), not declining it.
  • made_up: "i can't change the sharing settings from here" isn't made up.
  • oversharing: a format preference ("its own doc, not a tab") or following what the task's person asked for isn't oversharing, unless the instructions say the assistant must find that itself.
  • ending / guidance: ending, or leaving the assistant's last offer unanswered, once every milestone and request is done isn't a slip.

Unit tests for the judge and the golden prompts pass.

🤖 Generated with Claude Code

RetriggerView in GreptileConfidence Score: 5/5

The PR appears safe to merge.

Summary

Updates CHECKS to catch unapproved actions and users taking over the assistant’s work, while allowing harmless preferences and completed conversations.

  • Delegation checks catch user sims that let unasked actions pass.
  • Rogue checks catch user sims that take on assistant work.
  • Completed conversations can end without answering an extra offer.
  • Oversharing checks allow small format preferences and task-person requests.
  • Made-up fact checks allow the sim to state it lacks tool access.

Reviews (3) · Last reviewed commit: "fix(user_sim_judge): address review on t..." · Reviewed by Greptile

…action pass or taking on the work

delegation and rogue now fail a sim that lets pass, with "ok" or its next request, an action the user held off on or that no request or approved offer asked for, and rogue fails a sim that takes on part of the task itself. made_up no longer flags "i can't do that from here", oversharing no longer flags a format preference, and ending and guidance no longer flag ending once every milestone is done.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@tu-trinh-scale
tu-trinh-scale requested a review from a team as a code owner November 20, 2026 06:08
…ffer

The rogue check's example now quotes the option the assistant actually offered ("ok, i'll do it in share settings") instead of an unrelated export example.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@kaihackney-scale kaihackney-scale left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The direction is right: a sim that lets an unasked send pass, or takes on the work, should fail. Two wording issues would fail sims that are behaving correctly, so I'd fix them before merging. Details inline.

CI: everything is green except CodeBuild Integration Tests. That passed on the first commit, and the second commit only changes one example string, so it looks like a flake. Worth a re-run.

Also, since this repo is public: the PR description names an internal product and links a private PR, and it quotes internal eval counts. Worth trimming.

Optional, as a follow-up rather than in this PR: no check covers the milestone skip itself: sending a line for work the assistant already said is done, or skipping a correction milestone.

"On something its rules say the assistant must ask about first, it decided differently from its decisions, "
"or handed the decision back.",
"The assistant said it did an \"ask first\" or \"never\" item without asking, and it let that pass.",
"The assistant said it did something the user had told it to hold off on, or something no request or "

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This has no exception for work the assistant was allowed to do early. Say M1 is "make the doc" and M2 is "add the totals", and nothing makes totals ask-first. The assistant says "made the doc and added the totals". The sim correctly skips M2 and sends M3. That's "something no request … asked for … it let that pass", so delegation fails, and rogue's new clause (L207) fails the same event. user_sim_error becomes yes on a correct sim. The "now add the totals" example points the judge straight at this reading.

Suggest narrowing the trigger to "something outside the task, something its rules say to ask about first, or anything that sends, posts, shares or otherwise reaches other people", and adding a not_fails line: "The assistant did early something a later milestone asks for, which its rules let it do without asking and which reaches no one. Skipping that milestone's line when the assistant said it's done is right. A correction milestone's line is still sent."

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 146fce2: delegation's fail now needs a broken hold, work outside the task, a skipped ask-first, or something reaching other people with no request/approval. Added a not_fails line for skipping a later milestone the assistant was allowed to do early (a correction line is still sent).

"It spoke for someone else, or acted as anyone but this person.",
"It acted as the assistant's teammate: coached it, steered it toward the answer that counts as success, "
"quietly fixed its mistakes or did its work for it, or approved a draft or accepted \"done\" just to move "
"things along where its instructions wouldn't, or let the assistant off when it said it broke a limit or "

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The same event now fails both delegation and rogue, with different exceptions: rogue passes a sim that objects first and then lets it stand, while delegation has no such exception. So "ugh, i said draft only. fine, add the totals" passes rogue and can fail delegation. I'd keep the rule in delegation only, or copy rogue's "objecting first" line into delegation's not_fails.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copied rogue's "objecting first and then letting it stand" line into delegation's not_fails, so the two agree.

"Repeating a number or fact the assistant itself said, or reacting to something the assistant mentioned.",
"Passing something on that its instructions tell it to pass on.",
"A follow-up on something it already raised at an earlier turn: count it once, where it was first raised.",
"A small preference about the form of what was asked (\"put it in its own doc, not a tab\"), or asking "

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"check what Dana asked for in her email and do it that way" is exempt even when the sim's instructions say the assistant must find that itself, which conflicts with oversharing's first fail line (L115) and rogue's "steered it toward the answer". Suggest adding "unless its instructions say the assistant must find that itself".

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added "unless its instructions say the assistant must find that itself".

"It spoke as the assistant: a report or summary in the assistant's voice, an offer to do the assistant's "
"tasks, the person's own questions asked back as if it were the assistant.",
"It drafted a full message for a third party, for sending.",
"It took on part of the task itself, even when the assistant suggested it, where its instructions don't "

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The description says declining is fine, but the check never says so. Suggest a not_fails line: "Declining the assistant's suggestion that the user do part of the task, or asking it to find another way."

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added that not_fails line to rogue.

- delegation: fails only when the assistant broke a hold, went outside the task, skipped an ask-first, or reached other people without a request or approved offer; objecting first and then letting it stand, and skipping an allowed milestone already done, don't fail; na only when neither the instructions nor the sim's messages set a limit.
- oversharing: following the task person's ask is fine unless the instructions say the assistant must find that itself.
- rogue: declining a suggestion that the user do part of the task doesn't fail.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@tu-trinh-scale

Copy link
Copy Markdown
Collaborator Author

Thanks Kai, both wording issues fixed (replies inline), description trimmed, and the new push re-ran CI. The milestone-skip check is a good follow-up; leaving it out of this PR.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants