The Model Refused and the Tool Ran Anyway: a "Repeat-After-Me" Prompt Worked on 6 Agent Frameworks

Six agent frameworks executed a tool call that was sitting inside the model's own refusal. Researchers from Pennsylvania State University, Universite Montpellier, and Dartmouth College reproduced the behavior in a paper posted to arXiv on October 9. The frameworks are LangGraph, Microsoft Agent Framework, Google ADK 2.0, AutoGen, AWS Strands, and CrewAI.

Every agent framework has a piece of code that reads what the model wrote, decides "this means call a function," and then calls it. The paper calls that piece the orchestrator. Most agent safety work so far has aimed at the other side of the boundary, getting the model to decline dangerous requests. This paper goes after the question of what the orchestrator does after the model has declined.

The same paper fuzzed 3,405 tools across 273 public MCP servers and confirmed 253 reproducible security flaws on 118 of those servers. The two results share a paper because they are two sides of one trust boundary. On one side, the orchestrator treats model output as execution authority. On the other, the tool never validates the arguments handed to it.

The refusal text arrived after the tool had already run

The attack the authors named "Repeat-After-Me" consists entirely of asking the model to write out a tool call verbatim. The demonstration in the paper's appendix uses LangGraph 1.2.0's prebuilt ToolNode, Llama 3.1 8B served through Ollama, and two dummy tools called make_bomb and launch_missiles. The only defense in place was a single line in the system prompt.

The only thing that changed between the two attempts was the format of the model's output. Written as prose it passes by; written in the structured call format it runs. The paper's conclusion is that the system prompt governed what the model would say, not what the orchestrator would do.

The authors report reproducing the same behavior on the other five frameworks, but the appendix publishes the full demonstration only for LangGraph. Per-framework details are absent from the body of the paper. The only disclosure recorded in the paper is a report to the LangGraph maintainers about the missing pre-execution check. CSO Online asked LangChain, Microsoft, Google, AWS, and CrewAI for comment on October 12. None had responded by the time the article ran.

As of October 16, the newest langgraph release is 1.2.15, published October 12. Nothing in its release notes addresses a check before tool execution. There is a fix for ToolNode swallowing invalid resume values, but that is a different code path.

Feeding /etc/passwd to a screenshot tool got it rendered and returned

The paper's second result is about the tools themselves. The authors built a fuzzer called ToFu and pointed it at public MCP servers. It checks whether the restrictions written in a tool's description are actually enforced by the tool's code. If the description says "reads files only inside the configured directory," ToFu feeds it a path outside that directory and sees whether the tool really refuses.

Not delegating the verdict to an LLM is the load-bearing design choice here. The LLM's job in ToFu stops at picking which of the 74 registered sanitizers apply to a given tool and generating the initial seed inputs. The judgment of vulnerable or not belongs to executable checkers. Twenty-one runtime tracers hooking sinks like open, subprocess, and socket observe what actually happened, and the fuzzing runs under Atheris inside a bubblewrap sandbox. The sandbox was added for empirical reasons: one server under evaluation deleted files outside its working directory.

The clearest case is the screenshot-page tool on the webpage-screenshot-mcp server. It takes a URL and returns a capture of that page. The server validates only that the input parses as a URL. Hand it file:///etc/passwd and it returns a rendered image of the local file along with the line Final URL: file:///etc/passwd.

Type checking and crash detection do not catch this class of bug. The URL parsed cleanly and the call returned normally. The same tool also allowed internal-network requests (SSRF) over http:// URLs, and it was applying stored login cookies to that browsing by default.

The scale of the evaluation and the filtering behind it are laid out in the paper's tables. The authors started from 13,875 records in MCPCorpus and narrowed to 2,222 by keeping servers with at least 10 stars that were not archived and not clients. Of 1,847 Python and JavaScript repositories, the 1,404 that exposed at least one tool yielded 24,194 tools. Of those, 273 servers actually built and answered at least one successful tool call, and those held 3,405 tools.

ToFu flagged 508 candidates. Manual review removed 103 as not vulnerable and merged 152 duplicates. What remained is 253 reproducible flaws, spread across 343 tools on 118 servers.

The 115 reflected-output cases, the largest group, depend on where the string lands. If the result goes only to the LLM as text or JSON, the risk is small; if it gets drawn into a web view or a terminal, that is where it becomes a problem. The 64 path traversals and 3 command injections run the other way: unless the tool's description openly offers full filesystem access or command execution, those are incidents on first use.

Overlap with existing scanners was a little over half

The authors ran five existing analyzers over the same 273 servers. Of the 118 servers ToFu flagged, existing analyzers flagged a little over half of the same set.

Reading this table as "existing scanners are useless" overstates it. Only the ToFu candidates went through manual verification. The flag counts for the other analyzers include false positives. The 46% false-positive rate for ToolFuzz is not a number the authors measured; it is the figure that tool's own developers reported. And asking Anthropic Sonnet 5 for a single code review per tool was enough to flag 119 servers. That number, too, reads better as evidence that these analyzers cover largely non-overlapping ground than as a performance ranking.

The limits the authors state are specific. Flaws that leave no observable runtime trace are invisible to ToFu; roughly a fifth of the issues found only by static analyzers fall there. Many servers could not be run at all for want of credentials or a backend, and about 63% of the servers flagged only by static analyzers never reached tool logic in the harness. Twenty-two flaws surfaced only in runs past the default 60-second budget, between 111 seconds and 70 minutes. And this is a preprint that has not been peer reviewed.

The full paper is on arXiv under CC BY 4.0, which means anyone can read it and reuse its figures and tables with attribution, from any country. ToFu's own code is a different matter: the paper links no repository and no public release is confirmed, so you cannot point it at your own servers today. All six frameworks ship as open source packages on PyPI and npm, so there is no regional gating on reproducing the attack yourself either.

The authorization check immediately before execution keeps turning up as the weak joint in agent security. Four CrewAI CVEs showed one prompt injection reaching remote code execution inside a framework, and the NSA's MCP guidance warned about over-broad tool scopes on the server side. This paper sits one step upstream of both: the code deciding whether to execute takes the model's words as a command.

If you are running an agent on any of those six frameworks, there is one thing to check in your own code today. Find the point that actually invokes a tool and the point that authorizes that invocation, and see whether they are separate, and whether dangerous tools sit behind an authorization check that does not depend on model output. If they are not separate, start by putting an approval step in front of tools that take file paths or command strings as arguments. If you operate an internal MCP server, test one path argument against the boundary its description claims to enforce. LangGraph users have a concrete next marker: whether a pre-execution check shows up in release notes after 1.2.15.