In our previous article, Enterprise AI model selection: Balancing performance, privacy, and operational fit , we described the 4 layers that make up an effective enterprise AI architecture. In this article, we'll talk about who operates each layer, where workloads run (managed APIs, self-hosting, or hybrid), and how Red Hat AI Enterprise provides all 4 layers as a tightly integrated production AI system. We'll also cover some deployment implications for RAG , fine-tuning , and agents, and some considerations around Day 2 operations.
When you're building out your AI infrastructure, one fundamental choice is whether to use managed AI models via APIs or host the models yourself. That decision directly impacts the architecture we've already discussed , and affects cost structure, data privacy, operational complexity, and how quickly you can scale. Managed AI models are accessed through APIs from providers such as OpenAI, Anthropic, and Google. Requests go to their cloud infrastructure and responses come back, usually billed per token or per request. The provider operates infrastructure layers 1 through 3.
Your team chooses region, model name, tier, and quotas, but doesn't manage GPUs, weight storage, inference engines, or serving. The integration point is the vendor endpoint and API contract. That arrangement avoids capital expense and Day 2 operations for serving, but per-token cost can rise quickly at scale, and prompts and outputs typically leave your network unless the vendor offers private or regional terms. Availability, pricing, throttling, and model updates remain under provider control. Self-hosted models run on infrastructure you control, on premise or in a private cloud.
Your team operates layers 1 through 3, or delegates them to a managed Kubernetes or AI platform on your estate. Artifacts are downloaded or mirrored, inference engines are deployed behind a serving layer, and monitoring, scaling, patching, and model promotion stay in-house. Data and prompts can remain inside your boundary, which helps with regulated workloads. Infrastructure cost is often more predictable at high volume, but the trade-off is GPU capacity, platform skills, and incident response.