Red Hat just shipped RHEL AI 3.4.0 (build 20260602-1780387621) on AWS Marketplace. If you are already running RHEL AI on EC2, your existing instances are unaffected — but upgrading is recommended.
What is RHEL AI
Red Hat Enterprise Linux AI is a purpose-built operating system image that packages everything you need to run AI inference and fine-tuning workloads:
- InstructLab for synthetic data generation and model alignment
- vLLM for high-throughput inference serving
- Granite models pre-loaded and ready to serve
- GPU drivers (NVIDIA CUDA) pre-configured
- SELinux and container isolation enabled by default
It ships as an AMI on AWS Marketplace, a QCOW2 for on-prem, and ISOs for bare-metal deployment.
What Changed in 3.4
While Red Hat has not published detailed release notes for this specific build, version 3.4 follows the rapid cadence established since RHEL AI 1.1 in late 2024. Based on the Red Hat AI platform roadmap shared at Summit 2026:
- InstructLab updates — improved synthetic data generation pipelines and model evaluation tooling
- vLLM performance — latest upstream vLLM with better throughput on NVIDIA GPUs
- Security patches — RHEL 9 base layer with all current CVE fixes
- Granite model refresh — newer model checkpoints with improved reasoning capabilities
- bootc image mode — immutable OS updates via
bootc upgradefor atomic rollbacks
How to Upgrade on AWS
Option 1: Launch a New Instance
The simplest path — launch the new AMI and migrate your workload:
# Find the new RHEL AI 3.4 AMI from Marketplace
aws ec2 describe-images \
--owners aws-marketplace \
--filters "Name=name,Values=*RHEL-AI*3.4*" \
--query 'Images[].{ID:ImageId,Name:Name,Date:CreationDate}' \
--output tableOr visit the AWS Marketplace listing and click Launch.
Option 2: In-Place Upgrade with bootc
If your instance supports image mode (RHEL AI 1.3+):
# SSH into your running instance
ssh -i your-key.pem cloud-user@your-instance-ip
# Check current version
cat /etc/redhat-release
ilab --version
# Pull and stage the new image
sudo bootc upgrade
# Reboot to apply
sudo systemctl rebootAfter reboot, verify:
cat /etc/redhat-release
rpm -q vllm instructlabOption 3: Keep Running (No Action Required)
Your existing subscription and running instances are unaffected. Previous versions remain accessible. Only upgrade when your maintenance window allows.
SSH Access Reminder
RHEL AI on AWS uses:
- Username:
cloud-user(notec2-user, notroot) - Auth: SSH key pair you selected at launch
- Root: Disabled by default (use
sudo)
# Set permissions on your key
chmod 400 your-key.pem
# Connect
ssh -i your-key.pem cloud-user@ec2-xx-xx-xx-xx.compute.amazonaws.comInstance Sizing for RHEL AI
For inference workloads with Granite models:
| Instance Type | GPUs | VRAM | Best For |
|---|---|---|---|
g5.xlarge | 1x A10G | 24 GB | Small models, development |
g5.12xlarge | 4x A10G | 96 GB | Granite 7B inference |
p4d.24xlarge | 8x A100 | 320 GB | Fine-tuning, large models |
p5.48xlarge | 8x H100 | 640 GB | Production training |
Why This Matters
RHEL AI is Red Hat’s answer to the “AI platform fragility” problem. Instead of stitching together drivers, frameworks, and serving layers yourself, you get a tested, supported stack that upgrades atomically.
Combined with InstructLab fine-tuning and Ansible-driven deployments, version 3.4 continues the trajectory toward enterprise AI that is as manageable as any other RHEL workload.
