FFmpeg fixed a high-severity flaw in its MagicYUV video decoder that researchers dubbed PixelSmash, a bug tracked as CVE-2026-8461 that can trigger remote code execution on some Jellyfin servers and denial of service in apps such as Kodi, Emby, Nextcloud, PhotoPrism and OBS Studio.
KEY FACTS
- Bug type Heap out-of-bounds write in the MagicYUV decoder
- Severity CVE-2026-8461 scored 8.8
- Trigger Malicious AVI, MKV or MOV files can set it off
- Fix FFmpeg 8.1.2 addresses the issue
- Exposure Any app using libavcodec is considered vulnerable
A technical analysis from JFrog said the flaw comes from how MagicYUV handles slices, which are independently decoded regions of a video frame. The issue is described as a one-row heap buffer overflow caused by a mismatch between frame allocation and chroma plane height calculations.
Researchers said the bug can be triggered when a user opens a crafted video file, when a directory containing the file is browsed for thumbnail generation, or during automated media ingestion. They found that several media tools, including Kodi, OBS Studio, PhotoPrism and desktop thumbnail generators in GNOME, KDE and XFCE, use FFmpeg with the decoder enabled.
JFrog said it demonstrated remote code execution on a Jellyfin 10.11.9 media server through a normal media library scan. The report said the attack path involved a crafted MagicYUV AVI file, automatic metadata extraction by ffprobe and a hijacked buffer free call that led to system command execution as the service user. The researchers also said the exploit requires ASLR to be disabled or defeated, and that a separate FFmpeg information disclosure flaw could help bypass that protection.
Even where code execution is not possible, the disclosure said the vulnerability is enough to cause a denial-of-service on affected systems. Plex uses a custom FFmpeg build with decoders disabled and a minimal allowlist, which reduces the risk, while Jellyfin updated its bundled FFmpeg version and PhotoPrism is adding a file format blocklist. Nextcloud received the report through HackerOne but did not plan a fix because the issue is outside its codebase.
WHY IT MATTERS
The flaw has a wide attack surface because the decoder is present in many projects that rely on FFmpeg to process untrusted media safely. That makes the issue relevant not only to media servers, but also to file preview systems, chat platforms and other applications that handle uploaded video.