Platform Lessons That Could Tame Risky AI Agents

Adam Shostack once built secure platforms for a living. Now he watches companies hand powerful tools to large language model agents and hopes they remember the basics. His October 2024 piece in Communications of the ACM lays out a simple truth. Trust does not come free. It must be designed in from the start.

Shostack argues that developers treat these agents as helpful colleagues. They give them internet access, code execution rights, and the ability to act on behalf of users. The result looks efficient. Yet it ignores decades of hard-won security experience. Too much power. Too little oversight.

Platform security rests on a handful of ideas that still apply. Isolation keeps one faulty component from dragging down the rest. Least privilege means an agent gets exactly the rights it needs for one task and nothing more. Defense in depth adds layers so that a single failure does not open the floodgates. These are not new concepts. They protected operating systems, cloud workloads, and mobile apps. They can protect agents too.

But current agent frameworks often skip them. An agent might call external APIs, read private files, or spin up new processes without clear boundaries. One prompt injection or flawed tool call and sensitive data walks out the door. Or worse, the agent starts making decisions that affect production systems. Shostack compares it to running untrusted code in a hostile environment and expecting good behavior. The comparison stings because it is accurate.

Recent incidents show the gap widening. In early 2026 researchers tested frontier models on real-world vulnerabilities. Anthropic’s Claude Mythos Preview turned 157 flaws into working exploits even with standard mitigations enabled, according to a LinkedIn analysis of the ExploitGym benchmark. GPT-5.5 managed 120. Sandboxing helped but did not eliminate the threat. Agents simply found paths around the protections.

That same year the Cloud Security Alliance released guidance on applying zero-trust principles to large language model environments. The March 2026 report stresses that traditional perimeter defenses fail against dynamic, data-hungry agents. Instead, organizations must verify every action, segment workloads, and monitor non-human identities continuously. The CSA paper maps these controls across model, application, integration, and infrastructure layers.

OWASP updated its risk list for 2025 and followed with a 2026 agentic edition. Excessive agency sits near the top. It describes systems granted too much autonomy, allowed to chain actions without human checkpoints. One compromised memory store or poisoned tool description and the agent goes rogue. The OWASP Gen AI project now tracks these patterns across autonomous workflows. Their June 2026 governance report, referenced in industry newsletters, highlights the gap between early threat models and actual production failures.

Security teams have started to respond. Some run agents inside hardened Nix sandboxes that deny all access by default. Others treat every agent as a distinct identity with its own lifecycle, inventory entry, and permission set. A March 2026 analysis from Arctiq notes that misconfigured AI identities create new attack surfaces. Prompt injection remains dangerous, but tool abuse and privilege escalation pose equal threats when agents can call external services.

Orca Security’s June 2026 guide pushes for agentless visibility across cloud environments. The firm scans for shadow AI deployments, maps vector databases and inference endpoints, then scores risks in one dashboard. Their approach combines data security posture management with AI-specific controls. Least privilege and zero-trust appear again as foundational. The message echoes Shostack: assume nothing, verify everything.

Yet many enterprises still prioritize speed. They deploy agents that book meetings, query databases, and approve expenses with minimal guardrails. A SANS Institute course on GenAI and LLM application security now dedicates entire sections to agentic threats. Students learn to threat-model LangChain loops, secure memory stores, and enforce output validation. The curriculum reflects growing demand. Boards want AI initiatives, but they also want assurance that a rogue agent will not leak trade secrets or trigger compliance violations.

Microsoft has expanded its internal taxonomy of agent failure modes. The list includes tool misuse, excessive agency, memory contamination, and gaps in human override. Security teams use it to reason about deployed systems. It pairs well with runtime monitoring that baselines normal agent behavior and flags deviations. Continuous verification replaces one-time trust.

Researchers have synthesized hundreds of papers on the topic. One 2026 survey reviewed 247 works and organized threat surfaces around information flow, delegated authority, persistent state, and control-flow hijacking through tools. The conclusion feels familiar. Many defenses are non-compositional. They work in isolation but collapse when agents chain multiple steps across sessions.

So what does effective protection look like in practice? Start with clear boundaries. Place each agent in its own sandbox with network egress controls, file system restrictions, and audited API calls. Enforce least privilege at every layer. An agent that summarizes emails does not need database write access. Add defense in depth through input sanitization, output filtering, and human-in-the-loop checkpoints for high-impact actions.

Inventory matters too. Organizations cannot protect what they cannot see. Maintain a register of every deployed agent, its tools, its data sources, and its permission profile. Treat these non-human identities with the same rigor once reserved for service accounts. Rotate credentials. Monitor for anomalous patterns. Log every decision and action for later review.

Testing must evolve. Traditional red-team exercises now include agent-specific scenarios. Can the system resist prompt injection that tricks it into deleting records? Does it leak data when given ambiguous instructions? Will it chain tools in unsafe ways? ExploitGym-style benchmarks help quantify these risks before production rollout.

The financial sector offers early signals. Banks experiment with agents that analyze transactions but keep final approval with humans. Insurance firms let agents draft policies yet require underwriter sign-off. These hybrid models accept that full autonomy carries costs that outweigh benefits in regulated settings.

Shostack’s core advice still holds. Never ask a model to do something a deterministic system can handle reliably. Reserve agents for tasks that truly benefit from flexibility and natural language understanding. For everything else, stick with scripts, APIs, and rule-based automation. The distinction prevents over-reliance on systems whose behavior remains partly opaque.

That opacity fuels much of the worry. Even frontier models can surprise developers with unexpected chains of reasoning. Memory poisoning, where past interactions alter future outputs in subtle ways, adds another vector. Supply-chain attacks on tool libraries or training data create persistent risks that are hard to detect.

Yet progress appears. New protocols for agent communication include authentication and authorization steps. AI firewalls filter both inputs and planned actions. Some platforms now expose verifiable logs that let auditors reconstruct every decision. These features borrow directly from platform security playbooks written long before large language models existed.

The industry stands at a crossroads. Adoption races ahead. Security thinking struggles to keep pace. Companies that treat agents as untrusted code running in hostile territory will build lasting advantages. Those that assume good intent risk painful lessons. The old rules still apply. They simply need fresh application.

Recent guidance from Palo Alto Networks Unit 42, the US Cyber Security Institute, and Arthur AI all reach similar conclusions. Zero trust for agents is not optional. It forms the baseline. Continuous monitoring, strict access controls, and security-by-design principles turn experimental pilots into production systems that regulators and customers can accept.

Shostack ended his piece with a call to engineer trust rather than assume it. Eighteen months later the evidence supports him. Agents that book flights have caused real headaches. Agents that control infrastructure could cause far worse. The fixes exist. They come from the same toolbox security teams have used for decades. The question is whether organizations will open it before the damage mounts.