[ROCm][Quant] Minmax-M3: enable fp8_per_channel and fix SwiGLU-OAI fp8 MoE for bf16 weights on mi300x

[hongxiayang]

Purpose

To improve the bf16 model serving perf on MI3xx series by quantization.

The fp8 w8a8 MoE quant config dropped the SwiGLU-OAI alpha/beta that models
such as MiniMax-M3 pass to FusedMoE (swiglu_alpha=1.702, swiglu_beta=1.0).
Only swiglu_limit was forwarded, so the silu_and_mul_with_clamp kernel ran
with its default alpha=1.0/beta=0.0 and produced garbage (gsm8k 0.00) on both
the serialized (Fp8MoEMethod) and online (_Fp8OnlineMoEBase) fp8 MoE paths.

Plumb gemm1_alpha/gemm1_beta through the fp8 w8a8 MoE config chain:

• fp8_w8a8_moe_quant_config: add params, forward to FusedMoEQuantConfig.make
• make_fp8_moe_quant_config: forward them in the default/TRITON branch
• Fp8MoEMethod.get_fused_moe_quant_config: read layer.swiglu_alpha/beta
• _Fp8OnlineMoEBase.get_fused_moe_quant_config: same, for the online path

Also add "fp8_per_channel" to the ROCm supported_quantization allowlist. The
PTPC methods (Fp8PtpcOnline{Linear,MoE}Method) already exist and the ROCm
rowwise fp8 scaled-MM supports per-channel weight scales; the method was simply
not allowlisted on ROCm.
Verified on bf16 model with --quantization fp8_per_channel on MI300x.

vllm serve /Path-to-MiniMaxAI__MiniMax-M3__bf16
--tensor-parallel-size 8 --gpu-memory-utilization 0.90
--max-model-len 4096 --block-size 128
--attention-backend TRITON_ATTN --no-enable-prefix-caching
--quantization fp8_per_channel
--compilation-config "{"cudagraph_mode":"FULL_DECODE_ONLY"}"

Recommended flags:
VLLM_USE_BREAKABLE_CUDAGRAPH=0 VLLM_ROCM_USE_AITER=1 VLLM_ROCM_USE_AITER_MOE=0 + --quantization fp8_per_channel

Test Plan

Test Result

On MI300x, for bf16 weight: fp8_per_channel halves weight memory, gives +75% KV headroom and ~+22–28% high-batch (conc64) throughput at near-bf16 accuracy within noise.

Further enabled AITER linear further improved:

VLLM_USE_BREAKABLE_CUDAGRAPH=0 VLLM_ROCM_USE_AITER=1 VLLM_ROCM_USE_AITER_MOE=0 + --quantization fp8_per_channel

conc1 +6%
conc64 +5%, +28% over bf16

---

Essential Elements of an Effective PR Description Checklist

• The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
• The test plan, such as providing test command.
• The test results, such as pasting the results comparison before and after, or e2e results
• (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.

[hongxiayang]

cc @zyongye

[tjtanaa]

Let's land this only after this minimax M3 PR is merged #45381

Also @hongxiayang once the minimax model PR is landed, can you share the bf16 and this ptpc accuracy, rather than just with a statement stating there are not degradation? Thanks