Posted on October 21, 2026 by Roy Libman, CPO, Radar (Skyhook. io) If you're evaluating AI SRE products, you probably own most of one already. The coding agent your engineers use, whether that's Claude Code, Codex or Cursor, can already investigate a Kubernetes cluster. What it usually lacks is context: what's failing right now, and what changed before it broke. We measured how much that gap matters.

The same agent and model worked through 50 broken clusters twice, once with a shell and raw kubectl, and once with read-only MCP tools that return that context: With context the agent was slightly more accurate: 46 correct against 45, and a mean judge score of 0. Both arms were right on 44 of the same faults. The bigger difference was time, which is what counts during an active outage: nobody wants to wait fifteen minutes for an AI investigation while users are getting errors. With context, the agent had a correct diagnosis within two minutes on 86% of faults, against 60% with a shell.

With a shell, one fault in ten took more than ten minutes. The gap was widest on the 14 faults where every pod reported Ready but the application was broken: 326 seconds on average with a shell, 79 with context. Fewer tool calls also means fewer model turns, which usually means lower token spend. So before you evaluate an AI SRE, give the agent you already have that context, and measure it. That gives you a baseline for what the product adds. We ran 50 faults from SREGym , the open Kubernetes fault benchmark from Microsoft and UIUC, on a live 3-node Amazon EKS cluster.

Both arms used Claude Code on Claude Sonnet 5 through Amazon Bedrock. One had a shell and raw kubectl. The other had read-only MCP tools from Radar, an open-source Kubernetes UI and MCP server that we maintain, with kubectl blocked so it couldn't fall back to a shell. SREGym's judge, Claude Opus 5 at temperature 0, graded the first diagnosis each run submitted. Context didn't fix the model's judgment. Both arms missed the same three faults. In one, a mutating webhook rewrote the pod's memory limits at admission.