Blog
1554+ articles — Page 11 of 65
llm-d at Red Hat Summit 2026: KV-Cache Aware Routing for vLLM
Red Hat presented llm-d at Summit 2026: cache-aware load balancing for vLLM. Cold 4.3s vs warm 0.6s (7x faster), $0.30 vs $3.00 per 1M tokens with caching.
8 min read
Wide Expert Parallelism on NVL72: Scaling MoE Inference
How NVIDIA TensorRT-LLM Wide-EP distributes MoE experts across GB200 NVL72 racks for 1.8x per-GPU throughput on DeepSeek-R1 scale models.
5 min read
bootc Image Plus Config: Managing OS Configuration at Scale
How to combine bootc immutable OS images with runtime configuration for production Linux. Build the golden image, layer site-specific config, and update.
5 min read
Mistral Small 4 119B: One Model, Three Personalities
Mistral Small 4 unifies instruct, reasoning, and coding into a single 119B MoE model with 128 experts, 6.5B active per token, 256K context, and Apache 2.0.
5 min read
Red Hat Training Hub: Unified LLM Fine-Tuning at Summit 2026
Red Hat AI Innovation Team unveiled Training Hub at Red Hat Summit 2026 — a unified Python API for SFT, LoRA, OSFT, and GRPO fine-tuning with MLFlow.
4 min read
vLLM Recipes: Deploy Any Model on Any GPU
Community-maintained vLLM serve configurations for every major model on NVIDIA H100/H200/B200, Grace-Blackwell, and AMD MI300X/MI325X hardware.
4 min read
Red Hat Summit 2026 in Atlanta: Open Source Meets AI at
My experience at Red Hat Summit 2026 in Atlanta — from platinum sponsors to late-night labs, product spotlights, and Turkish Airlines scaling 60+ AI.
28 min read
Fix Astro Build Out-of-Memory on GitHub Actions: Bump
Astro builds with heavy image optimization can OOM on GitHub Actions runners with exit code 134. Set NODE_OPTIONS=--max-old-space-size=6144 on the build.
3 min read
Free Practical RHEL AI Book at Red Hat Summit 2026
I am giving away copies of my latest book Practical RHEL AI at Red Hat Summit 2026 in Atlanta. Meet me at the Lawn Lounge at 16:45 to grab your free copy.
4 min read
Checkmarx at DevWorld 2026: AI Is Not Enough for AppSec
Checkmarx presented a compelling case at DevWorld Conference 2026: AI-only security scanning creates noise, not safety. The hybrid approach combining.
5 min read
CodeMind Edge: Search Your Codebase with AI — No Cloud
CodeMind Edge is a local, privacy-first semantic code search engine built with Qdrant Edge. Ask plain-English questions and find relevant code instantly.
3 min read
DevWorld 2026: The Next Generation of AI Is Powered by
A DevWorld 2026 keynote argued that the next generation of AI depends on infrastructure, not models — six pillars from latency to 100K req/s scale.
2 min read
DevWorld Conference 2026: Stripe's Machine Payment
Key takeaways from DevWorld Conference 2026 featuring Stripe's Machine Payment Protocol (MPP) for AI agent commerce and Tailscale's vision of an AI.
8 min read
IPv8: The IETF Draft That Could Replace Both IPv4 and IPv6
Internet Protocol Version 8 is a managed suite that is 100% backward compatible with IPv4, eliminates NAT, and unifies modern routing.
5 min read
LangWatch Scenario: Stop Vibes-Testing Your AI Agents
At an Amsterdam meetup, the LangWatch team demonstrated Scenario — behavioral testing for AI agents. From vibes-driven pray-and-push to scenario-driven.
4 min read
GitHub Copilot Dev Days Amsterdam: Agentic SDLC and the
Recap of GitHub Copilot Dev Days Amsterdam at TrueFullstaq — live demos of Agentic SDLC, AI-assisted code reviews, incident triage, and the big question.
2 min read
KyvernoCon Virtual 2026: Policy-as-Code Meets AI and
Full agenda and key takeaways from KyvernoCon Virtual 2026 — covering CEL policies, AI workload provenance, ISO27001 compliance, multi-tenant RBAC, GitOps.
5 min read
Distributed Inference vs Multi-GPU Inference
A practical comparison of multi-GPU inference within a single node versus distributed inference across multiple nodes. Covers tensor parallelism, pipeline.
8 min read
Book Review: Architecting Generative AI Applications by
A review of Architecting Generative AI Applications (Packt, 2026) covering LLMOps, RAG, agentic systems, and closing the demo-to-production gap.
4 min read
FIO Storage Benchmarks for AI Training: NFS and SMB on
How to benchmark shared storage performance for distributed AI training workloads using FIO on Kubernetes. Covers NFS vs SMB throughput, IOPS patterns.
5 min read
Fine-Tuning Mistral with FSDP and LoRA on OpenShift
A production guide to fine-tuning Mistral models using FSDP2, LoRA/PEFT, and Run:ai on OpenShift with InfiniBand RDMA. Covers NCCL configuration, memory.
12 min read
GenAI-Perf: Benchmarking LLM Inference with NVIDIA's
How to benchmark vLLM inference endpoints using NVIDIA GenAI-Perf from multiple network locations. Covers token throughput, latency percentiles.
5 min read
Multi-Node Distributed Inference with LeaderWorkerSet on
Deploy large language models across multiple Kubernetes nodes using the LeaderWorkerSet (LWS) operator. Covers vLLM pipeline parallelism, NCCL over.
5 min read
Training RetinaNet with DDP on Run:ai: Multi-Node Object
A production guide to training RetinaNet object detection models using PyTorch DDP on Run:ai with Open Images Dataset V7. Covers multi-node orchestration.
6 min read