Fix cAdvisor + Docker 29 containerd-snapshotter Errors

The Error

You are looking at logs from a host running Docker 29 (with its new containerd-snapshotter storage driver) and cAdvisor as a metrics collector. Every minute — sometimes every few seconds — you see a flood of:

E0903 12:53:54.608044       1 manager.go:1116] Failed to create existing container:
  /system.slice/docker-f99d12ec59d70c79169fb8fa9802bd7c8ab5e25c5adc096acfa0d43dfca345d7.scope:
  failed to identify the read-write layer ID for container
  "f99d12ec59d70c79169fb8fa9802bd7c8ab5e25c5adc096acfa0d43dfca345d7".
  - open /rootfs/var/lib/docker/image/overlayfs/layerdb/mounts/f99d12ec59d70c79169fb8fa9802bd7c8ab5e25c5adc096acfa0d43dfca345d7/mount-id:
  no such file or directory

The same container IDs repeat across timestamps — 12:53:54, then 12:54:54, then 12:55:54 — producing identical errors. Your monitoring stack (Loki, etc.) fills up with noise.

Root Cause

This is not a corrupted layerdb or a crashed daemon. The mount-id file doesn’t exist because Docker 29 no longer stores container layers at that path. Here is the actual chain of events:

1. cAdvisor chooses the Docker factory

When cAdvisor starts and finds a Docker socket at /var/run/docker.sock, it prefers the Docker factory over its native containerd factory. This is the intended priority order in cAdvisor’s manager.go — Docker gets first claim on any container it can see.

2. Docker 29 uses containerd-snapshotter

Docker 29 introduced the containerd-snapshotter storage layout as part of its migration to a fully containerd-backed engine. Under this layout, the classic overlayfs layer store that cAdvisor’s Docker factory expects:

/var/lib/docker/image/overlayfs/layerdb/mounts/<id>/mount-id

…no longer exists. Docker has moved to a containerd-native snapshot store, and the mount-id files that the legacy Docker image driver created are gone.

3. The Docker factory is fundamentally broken against the new layout

cAdvisor’s Docker factory walks the legacy layerdb directory to map a container ID to its read-write overlayfs mount. When it hits the containerd-snapshotter layout and finds no mount-id file, it returns the “failed to identify the read-write layer ID” error. The same container IDs are re-checked on every reconciliation cycle (every ~1 minute), so the errors repeat indefinitely.

4. There is no clean flag to disable the Docker factory

This is the key gotcha: cAdvisor has no runtime flag to tell it “prefer the containerd factory even though the Docker socket is visible.” The Docker factory always wins. You can’t flip a config switch and keep cAdvisor running against Docker 29.

This is a genuine upstream compatibility gap, not a configuration mistake. cAdvisor’s Docker factory has not caught up to Docker 29’s containerd-snapshotter storage layout.

Why the Obvious Fixes Don’t Work

The Fix

Since cAdvisor produces zero useful data in this state (its Docker factory can’t read layers, and there is no flag to make it use the containerd factory instead), the cleanest production fix is to remove cAdvisor:

Option A: Remove cAdvisor (recommended)

# If running as a systemd service:
sudo systemctl stop cadvisor
sudo systemctl disable cadvisor
sudo rm /etc/systemd/system/cadvisor.service
sudo rm /usr/local/bin/cadvisor  # or wherever it was installed
sudo systemctl daemon-reload

# If running as a Docker container:
docker rm -f cadvisor
docker rmi gcr.io/cadvisor/cadvisor:latest

# If running as a Kubernetes Deployment:
kubectl delete deployment cadvisor
kubectl delete service cadvisor
kubectl delete daemonset cadvisor  # if deployed as DaemonSet

Option B: Switch to native Docker stats (if you need container metrics)

Use Docker’s built-in stats endpoint instead — it is containerd-aware:

# Real-time stats for all containers
docker stats --no-stream

# Or via the Docker API for programmatic use
curl --unix-socket /var/run/docker.sock \
  http://localhost/containers/json?stats=1

Option C: Upgrade cAdvisor (if a compatible version exists)

Check whether a cAdvisor release supports Docker 29’s containerd-snapshotter layout. Until one does, Options A or B are your safe paths.

# Check your versions
docker version    # should show 29.x
cadvisor --version

# Track releases: https://github.com/google/cadvisor/releases

Verify the Errors Are Gone

# Wait 2–3 minutes for cAdvisor's next reconciliation cycle
sudo journalctl -u cadvisor -f --since "3 min ago" | grep -i "mount-id"
grep -i "mount-id" /var/log/containers/cadvisor*.log 2>/dev/null

No new lines means the flood has stopped. Confirm your containers are still healthy:

docker ps          # or: kubectl get pods -A

Prevention

• Before upgrading to Docker 29, check whether your metrics stack (cAdvisor, Prometheus Docker exporter, etc.) is compatible with the containerd-snapshotter storage driver. Many tools that introspect the legacy layerdb path break silently — they just stop collecting data and start spamming error logs.
• Prefer containerd-native metrics going forward: the containerd metrics endpoint and ctr / crictl commands are stable across layout changes.
• Use the Docker stats API for container-level metrics on Docker 29 — it abstracts the storage driver so you don’t depend on the on-disk layout.

Related Guides

• Install containerd on Ubuntu (Kubernetes Runtime)
• Fix Docker Error: Overlay2 Driver Not Supported
• Fix Docker Error: containerd Not Running
• Fix Kubernetes Failed Create Pod Sandbox: CNI
• Docker Cheat Sheet 2026

Having trouble with your container infrastructure? Book a consultation with Luca Berton — Docker Captain and Production AI expert — to debug and optimize your production environment.