As enterprises move to autonomous agentic pipelines, 2 critical questions emerge: Can I add security to my agents without killing performance? And, Where should I focus optimization effort to get the most out of my agentic system? Most inference benchmarks test the model in isolation—a raw request to vLLM , a prompt in, tokens out. But production agentic systems don't work that way: Every request passes through an agentic harness that assembles context, manages sessions, injects tool schemas, and optionally provisions a secure sandbox for code execution, all before a single token is generated.
Understanding where latency lives across this pipeline is essential for capacity planning, deployment sizing, and knowing where optimization effort actually pays off. This is Part 1 of a 3-part series on benchmarking agentic AI performance on Red Hat OpenShift AI . Here’s what’s in store in future installments: In this post, we’ll demonstrate that sandbox security adds negligible overhead—minimal resource consumption and under 5% of end-to-end latency —making isolation a default-on decision rather than a performance trade-off.
We’ll also show that the GPU has significant spare capacity even at 10 concurrent agents, revealing that the performance optimization opportunity lies in the agentic harness pipeline, not necessarily in faster hardware. This means that, for engineering teams deploying agentic AI on OpenShift AI, meaningful performance gains will come from understanding the agentic pipeline and right-sizing your cluster for the workload. We share our methodology, tooling, and results so that engineering teams can apply the same approach to benchmark and size their own agentic deployments on OpenShift AI.
We benchmarked 2 progressively more complex configurations of the agentic AI pipeline, as laid out in Table 1. This layered design lets us isolate and attribute performance overhead to each individual component, rather than measuring the full pipeline as a black box.