Some conversations stick with you not because of a single quote, but because of the shape of the curve they describe. That was Pete Cheslock on the show floor at Red Hat Summit 2026 in Atlanta. Pete is active in the vLLM and llm-d communities out of Red Hat, and within a few minutes he had sketched an adoption timeline that matched almost everything else I saw at this Summit: enterprise AI did not creep forward this year, it jumped.
The One-Year Leap: RAG to Roll-Your-Own Models
Pete’s framing was blunt: at Summit last year, the teams he talked to were still figuring out RAG (Retrieval-Augmented Generation) — wiring a vector store to a hosted model API and calling it a pipeline. This year, the same people, often the same companies, are running inference infrastructure and training their own models. Not a different cohort further along the curve — the identical teams, twelve months later.
That distinction matters more than it sounds. RAG-as-first-step is low commitment: you keep someone else’s model, someone else’s weights, someone else’s GPU bill, and you bolt retrieval on top. Running your own inference stack and training your own models is the opposite of low commitment — it means owning GPU capacity planning, serving infrastructure, and the model lifecycle itself. Pete’s point was that a huge number of enterprise teams crossed that commitment threshold in a single year, which is a much faster maturity curve than most infrastructure shifts get.
From Prompting to Training: What “Own Models” Actually Means
It is worth being precise about what “training their own models” means in this context, because it is not fine-tuning a chatbot prompt. It is teams standing up the full loop: serving infrastructure for inference, and a training path back to improve the model on their own data. I saw the production version of exactly this pattern elsewhere on the Summit floor — a German public sector deployment that used a teacher-student fine-tuning approach to get a 7-billion-parameter open model to outperform GPT-4.1 on a narrow task, running on three GPUs. That is the practical destination of the curve Pete described: teams that started with RAG a year ago are now running that kind of pipeline themselves.
vLLM and llm-d: Where Pete Spends His Open Source Time
The two projects Pete is active in are not a coincidence — they are the two layers that make “serve your own models at scale” tractable without a hyperscaler-sized platform team behind you. vLLM is the inference engine: the thing that actually runs the model efficiently, with the throughput and memory tricks that make self-hosted serving viable in the first place. llm-d is the layer above it — a Kubernetes-native way to orchestrate that serving across a fleet of pods rather than a single instance. I wrote up the mechanics of llm-d’s KV-cache-aware routing in more detail elsewhere from a separate Summit session — Pete’s contribution is on the community side, helping the next wave of teams figure out how to go from a single vLLM pod to a fleet that behaves like a coherent inference platform, which is exactly the gap between “we tried RAG” and “we serve our own models.”
More GPUs, Better Tokens: The Infrastructure Bet
Pete’s stated bet was simple and worth sitting with: more GPUs means better token throughput, and deploying inference at scale is the name of the game. That sounds almost too obvious to write down, except it has a direct implication for anyone planning an AI infrastructure budget in 2026 — the constraint on your AI roadmap is not really the model anymore, it is how much GPU capacity you can provision and how efficiently your serving layer uses it. Teams that treat inference infrastructure as an afterthought to a model choice are optimizing the wrong variable. The organizations Pete is helping are the ones who have figured out that vLLM plus llm-d is how you turn a GPU budget into token throughput instead of leaving capacity idle behind a cache-blind load balancer.
Where This Leaves Enterprise AI Teams
What struck me most, hearing this from Pete right after walking past Omdia’s stat that 70% of AI proof-of-concepts never reach production, is how well the two data points fit together. The teams stuck at the PoC stage are the ones still doing RAG-as-a-demo. The teams Pete described — the ones now running inference and training their own models — are the ones that treated infrastructure as the real project, not the model. Open source inference tooling is what makes that jump affordable enough for a normal enterprise team to make in a single year instead of three.
Related Reading
- Red Hat Summit 2026 in Atlanta: Open Source Meets AI
- llm-d at Red Hat Summit 2026: KV-Cache Aware Routing for vLLM
- llm-d Joins the CNCF: Kubernetes-Native Distributed LLM Inference
- vLLM Inference Optimization on RHEL AI
- RAG Implementation Patterns on RHEL AI
About the Author
I am Luca Berton, AI and Cloud Advisor. I work at the intersection of platform engineering, cloud security, and enterprise AI deployments. Book a consultation.
