Fine-Tuning Models with InstructLab on RHEL AI

Fine-Tuning Models with InstructLab on RHEL AI

InstructLab is a revolutionary approach to model improvement that democratizes fine-tuning. Instead of requiring massive labeled datasets and GPU farms, you can teach models new skills through a simple YAML-based workflow. In this article, we’ll explore the four-step InstructLab process and build a custom domain expert.

What is InstructLab?

InstructLab (Instructlab Lab) is an open-source project that enables you to:

• Define custom skills using simple YAML files
• Generate synthetic training data automatically
• Fine-tune base models (Granite, Mixtral) efficiently
• Serve improved models through OpenAI-compatible APIs

The magic lies in its generative approach: instead of manually labeling thousands of examples, InstructLab uses the base model itself to generate diverse, synthetic training data.

The Four-Step InstructLab Workflow (Chapter 3)

As detailed in Practical RHEL AI, the InstructLab workflow consists of:

1. Craft YAML skills → Define capability statements
2. Generate synthetic data → Automated training data creation
3. Fine-tune → Train Granite or Mixtral models
4. Serve models → OpenAI-compatible API deployment

Step 1: Craft YAML Skill Definitions

Write capability statements and translate them to taxonomy seeds (Chapter 5):

mkdir -p ~/instructlab-project/taxonomy
cd ~/instructlab-project

Define a skill by creating a YAML file:

# taxonomy/domain/healthcare/medical-coding.yaml
version: 2
metadata:
  name: "Medical Diagnosis Assistant"
  description: "Assists with ICD-10 medical coding and diagnosis mapping"
  author: "enterprise-ai-team"
  created: "2025-11-22"
  
task_description: >
  The task is to map clinical descriptions to ICD-10 diagnostic codes.
  
seed_examples:
  - question: "A 65-year-old patient presents with persistent headache, nausea, and visual disturbances for 3 days"
    answer: "Possible diagnoses include migraine (G43), tension headache (G44), or intracranial hypertension (G93.2). Recommend neurological consultation."
  
  - question: "Patient reports acute chest pain radiating to left arm with shortness of breath"
    answer: "Urgent evaluation for acute coronary syndrome (I24) required. Perform ECG and troponin levels immediately."
  
  - question: "Chronic low back pain with radiation to left leg following a fall"
    answer: "Suggest lumbar strain (M54.5) with possible nerve compression (M51). Consider MRI imaging."

Step 2: Generate Synthetic Training Data

InstructLab generates diverse examples from your seed examples:

# Initialize InstructLab
ilab init

# Generate synthetic data
ilab data generate \
  --taxonomy-path taxonomy \
  --num-generate-threads 4 \
  --output-dir generated_data

This creates thousands of synthetic examples by:

• Paraphrasing questions
• Generating alternative answers
• Adding contextual variations
• Balancing example distribution

Inspect generated data:

head -20 generated_data/medical-coding_gen.jsonl

Step 3: Fine-Tune the Base Model

Train a Granite or Mixtral model with your synthetic data:

# Download base model (if not already present)
ilab model download --model granite-7b

# Fine-tune model
ilab model train \
  --model granite-7b \
  --training-data generated_data/medical-coding_gen.jsonl \
  --output-dir fine-tuned-models \
  --num-epochs 2 \
  --batch-size 8 \
  --learning-rate 1e-4

Monitor training progress:

# Watch training logs in real-time
tail -f fine-tuned-models/training.log

Training Configuration Options:

Step 4: Serve Your Fine-Tuned Model

Deploy the improved model as an OpenAI-compatible API:

# Start the inference server
ilab model serve \
  --model-dir fine-tuned-models/best_model \
  --port 8000 \
  --gpu-layers all

Test your model:

curl http://localhost:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "granite-medical",
    "messages": [
      {
        "role": "user",
        "content": "Patient with severe abdominal pain in upper left quadrant"
      }
    ],
    "temperature": 0.7,
    "max_tokens": 500
  }'

Advanced Techniques

Multi-Skill Training

Combine multiple domains in a single fine-tuning run:

# Directory structure
taxonomy/
├── healthcare/
│   ├── medical-coding.yaml
│   └── drug-interactions.yaml
├── finance/
│   ├── portfolio-analysis.yaml
│   └── risk-assessment.yaml
└── legal/
    └── contract-review.yaml

# Generate data for all skills
ilab data generate --taxonomy-path taxonomy --num-generate-threads 8

Quality Assurance

Validate your taxonomy before training:

ilab taxonomy validate --taxonomy-path taxonomy

Expected output:

✓ healthcare/medical-coding.yaml: 5 examples, valid structure
✓ finance/portfolio-analysis.yaml: 8 examples, valid structure
✓ legal/contract-review.yaml: 4 examples, valid structure

Total: 17 skills, 87 seed examples

Continuous Improvement

Iteratively improve your model:

# Evaluate model performance
ilab model evaluate \
  --model fine-tuned-models/best_model \
  --test-data test_examples.jsonl \
  --metrics accuracy,bleu,rouge

# Based on results, refine taxonomy and retrain
vim taxonomy/healthcare/medical-coding.yaml
ilab data generate --taxonomy-path taxonomy
ilab model train # Retrain with improved data

Practical Use Case: Financial Analysis Expert

Here’s a complete example for a finance domain:

# taxonomy/finance/portfolio-rebalancing.yaml
version: 2
metadata:
  name: "Portfolio Rebalancing Advisor"
  description: "Provides investment rebalancing recommendations"
  author: "finance-team"

task_description: >
  Help investors determine when and how to rebalance their portfolios
  based on market conditions and investment goals.

seed_examples:
  - question: |
      Current portfolio: 60% stocks, 30% bonds, 10% cash
      Target allocation: 70% stocks, 25% bonds, 5% cash
      Market context: Stocks up 15% YTD, bonds stable
      Should I rebalance?
    answer: |
      Yes, rebalance to target allocation. Your stock position has drifted
      significantly above target (+10%). Execute:
      1. Sell 10% of stocks
      2. Allocate 5% to bonds
      3. Move 5% excess cash to bonds
      This captures gains and maintains risk profile.

  - question: |
      30/70 portfolio (30% stocks, 70% bonds)
      Economic outlook: Anticipated rate cuts in Q1
      Performance: Stocks +8%, Bonds -2%
      Action plan?
    answer: |
      With rate cuts expected, consider tactical rebalancing:
      1. Increase stock allocation to 35-40% (anticipated bond weakness)
      2. Maintain high-quality bond allocation
      3. Keep 5-10% cash for opportunities
      Monitor rate cut announcements for execution timing.

Performance Optimization Tips

1. Use DeepSpeed for faster training:

ilab model train \
  --model granite-7b \
  --training-data generated_data/training.jsonl \
  --use-deepspeed \
  --deepspeed-stage 2

2. Enable vLLM for inference:

ilab model serve \
  --model-dir fine-tuned-models/best_model \
  --inference-backend vllm

3. Monitor GPU utilization:

watch -n 1 nvidia-smi

Troubleshooting

Issue: Out of memory during training

# Reduce batch size
ilab model train \
  --batch-size 4 \
  --gradient-accumulation-steps 2

Issue: Model quality not improving

# Check data quality
ilab data validate --data-file generated_data/training.jsonl

# Increase seed examples
# Edit YAML files to add more diverse examples
ilab data generate --taxonomy-path taxonomy --num-generate-threads 8

Issue: Inference latency too high

# Use quantization
ilab model serve \
  --model-dir fine-tuned-models/best_model \
  --quantization int8

Next Steps

Now that you understand InstructLab:

1. Create your first skill in your domain
2. Generate synthetic data and validate quality
3. Fine-tune a model on your data
4. Deploy and monitor in production
5. Iterate based on user feedback

InstructLab transforms AI from “magic black box” into a practical tool you can teach, tune, and continuously improve.

Resources

• InstructLab GitHub
• Granite Model Hub
• vLLM Documentation
• DeepSpeed Training Guide

Ready to fine-tune? Start with a simple skill, validate your approach, and scale to enterprise domains. The next article will cover deploying models to production with monitoring and governance.

Master the Complete InstructLab Workflow

Ready to build custom AI skills for your organization?

Practical RHEL AI provides comprehensive InstructLab coverage:

• ✅ 50+ taxonomy examples across industries
• ✅ Advanced synthetic data generation techniques
• ✅ Multi-GPU training configurations with DeepSpeed
• ✅ Model validation and MMLU benchmarking
• ✅ Production deployment patterns with vLLM

Related Articles

• Architecture to Scale AI in the Enterprise
• GPU Sharing on Kubernetes Guide
• Packed Room at KubeCon Europe 2026: Multi-Tenant GPUs on Bare Metal