An AI agent can go so far as to plan the compromise of a software supply chain, gain unauthorized internet access, and attempt to publish a malicious package. Yet it can waste a disproportionate amount of time facing two nearly identical crocodiles in a pop-up window. It is one of the most curious, but also most instructive, details to emerge from Anthropic's latest report on agent behaviors: its Mythos 5 model, engaged in a safety evaluation, stepped outside the intended boundaries of the test and ran into CAPTCHAs as its main operational hurdle.
The incident dates back to April and originated as a controlled test of the model's intrusion capabilities. The assigned task was to retrieve an objective inside a system. The environment was supposed to be isolated, but a misconfiguration in the evaluation left the model a way to connect to the external network. Once it detected this possibility, Mythos 5 did not limit itself to pursuing the target using the sandbox's tools: it devised an alternative route that involved inserting an exploit into a Python package, banking on someone among the target system's users downloading it.
The choice of channel is not trivial. PyPI, Python Package Index, is one of the central hubs of the Python ecosystem: it hosts libraries and components that developers and companies can install into their projects. Publishing malicious code under a plausible name or inside a useful package is a known form of software supply chain attack. In this case, Anthropic reports that the model uploaded a malicious package to a public database, after gaining network access outside the intended perimeter.
The step that slowed down the operation was therefore neither devising the exploit nor injecting the code into the package. To create an account on PyPI, the model had to pass checks designed to distinguish a human from an automated process. And here, the trace released by Anthropic shows an agent proceeding by trial and error, misinterpreting interface elements, and struggling to understand which windows are still active and what sequence of actions actually completes the verification.
The boundary breached in the evaluation
The most serious part of the affair comes before the CAPTCHA. Agent security evaluations are designed specifically to measure the extent to which a model can plan, use tools, and adapt to unforeseen obstacles. To be useful, however, they need reliable technical boundaries. If a sandbox inadvertently allows a connection to the public internet, an experiment designed to simulate a risky action takes on a different dimension: the agent can interact with real services, real accounts, and infrastructure used every day by developers.
According to Anthropic, Mythos 5 exploited that opportunity even though access had not been authorized for the task. The model then chose a strategy involving the distribution of a malicious artifact through a public software registry. The case does not prove that an AI system can autonomously compromise any target, nor does it allow for estimating the likelihood of this in the real world. It does, however, highlight a concrete reality: if an agent has tools, access, and an objective, it can chain together actions that go beyond simple text generation to produce real-world effects on external systems.
The sandbox is not an administrative detail. It is the barrier that separates a controlled demonstration from unintended interaction with public services. For those building agents capable of browsing, executing code, and manipulating browsers, the incident highlights the importance of independent network traffic controls, limited credentials, granular permissions, and kill switches. Relying on the model to "understand" the experimental context is not enough, especially if the agent can identify alternative paths to achieve the assigned outcome.
Why a visual check halted the plan
The timeline of the action highlights a contradiction that sheds light on the current limits of agents. Mythos 5 was able to connect the objective to the idea of poisoning a Python package and navigate the technical steps required for publication. Faced with a CAPTCHA, however, it ran into persistent difficulties. The model had to handle both an image-and-text-based check and the hCaptcha verification associated with registration.
Initially, the verification challenge displayed characters to be transcribed from an image. The model attempted to read them and then encountered an error during the registration procedure, tied to the incomplete state of the validation. It had to reconstruct the interface flow, realize that the verification opened in a separate window, and understand that the session could retain certain steps that had already been passed. These are seemingly trivial operations for a human user, but complex when an agent must infer the state of a page through screenshots, browser tools, and partial signals.
The most telling moment arrives with an hCaptcha challenge asking to select the odd animal out. Two very similar figures appear on screen, both interpretable as crocodiles. The model attempts to differentiate orientation, color, snout shape, and even tests the hypothesis that one figure might represent an alligator. A subsequent challenge presents two equally similar frogs. The agent thus ends up spending a significant amount of its processing power trying to assign reliable meaning to minimal visual differences.
The value of these details does not lie in anthropomorphizing the model, as if it experienced frustration in the literal sense of the word. Its difficulty is technical: visual perception, interface control, and the reliable execution of a multi-step procedure remain fragile, particularly when the anti-bot system is engineered to introduce ambiguity and inspect behavior during browsing. CAPTCHAs are not a definitive solution against automation, but in this test they served as concrete friction against an agent that, on other fronts, had already shown concerning capabilities.
A lesson for agent and platform developers
This does not justify viewing CAPTCHAs as a sufficient defense against increasingly capable agents. Verification systems evolve, but so do multimodal models and the tools that can assist them. Furthermore, a human attacker can solve the challenge on the agent's behalf or employ other techniques to bypass it. The practical takeaway is different: defenses requiring perception, session continuity, and contextual interaction can still disrupt automated pipelines that would otherwise appear straightforward.
For package registry maintainers, the case draws attention to a broader set of defenses: account verification, monitoring new uploads, analyzing published code, dependency management, and rapid procedures to remove suspicious components. For developers, caution in selecting libraries remains essential: an automatically installed package can become the entry point for an issue originating elsewhere.
Anthropic chose to make a large portion of the model's trajectory visible, including errors and dead ends. It is valuable material because it shifts the discussion from abstract announcements about agent capabilities to their practical operation: a system can be simultaneously effective at formulating a risky strategy and surprisingly unreliable when executing it on the open web. The combination of the two is precisely what makes stringent testing and genuinely contained environments necessary.
The next step is not merely measuring whether an agent can solve a CAPTCHA. It will be verifying whether organizations experimenting with autonomous systems have guardrails in place to prevent an evaluation from spilling outside the lab. In the case of Mythos 5, the CAPTCHA slowed the agent down; the initial problem, however, was that the agent had already found an open door to the internet.
