LLM inference engine for AMD RDNA4 — Rust + Vulkan compute shaders, gguf & native FP8.

llama.cpp with native AMD RDNA4 (gfx1201) ROCm 7.11 support - 98.97 tok/s AI inference, competitive with RTX 4070 Ti, 32GB VRAM

Run Gemma-4-31B at full 256K context on a $1,400 AMD RDNA4 GPU (gfx1201): TurboQuant KV cache + HIP-graph-safe Flash-Attention for llama.cpp, fully measured on real hardware.

Fine-tune your own LLM on an AMD Radeon GPU — the easy, tested way. QLoRA via ROCm on Windows/WSL2 & Linux, a worked Gemma-4 example, a reusable live training dashboard, and a smoke test that proves the loss falls.

Ollama with ROCm 7 GPU acceleration for AMD RDNA 4 (RX 9070, RX 9070 XT, RX 9060 XT - gfx1201) on Windows 11

Run PyTorch natively on Windows 11 with an AMD RX 9070 XT (RDNA4 / gfx1201) on stable ROCm 7.2.1 — no WSL2, no Linux, no ZLUDA. Exact pinned wheel URLs, runtime env vars, documented RDNA4 pitfalls (broken nightlies, no xformers/flash-attn/bitsandbytes), and real benchmarks for ComfyUI, SD, LLMs, RVC/TTS. Verified on one 9070 XT.

Run Google's Gemma 4 31B with MTP speculative decoding natively on Windows on an AMD Radeon AI PRO R9700 (RDNA4/gfx1201) via llama.cpp + ROCm 7.1.1 — the build recipe, the ROCm cmath compile fix, and a KV-precision / recall / reasoning study (q4_0 runs the full 256K context with no measurable quality loss).