[rollout, vllm] feat: KV-cache-aware request load balancer#7115

Draft

touch869 wants to merge 15 commits into

verl-project:mainfrom

touch869:router

Commits on Jul 22, 2026

[rollout] feat: extract load balancing router strategy

ZOULQ
authored and
hangangqiang
committed
suuport kv-event in multi replica

yyyyrf
authored and
hangangqiang
committed
[rollout] refactor: pluggable router strategy package + construction contract

zhaizhiqiangA
authored and
hangangqiang
committed
[rollout] feat(router): register kvcaware strategy + balancer skeleton

zhaizhiqiangA
authored and
hangangqiang
committed
[rollout] feat(router): kvcaware store impl + prefix-hash utils
hangangqiang
committed
[rollout] feat(router): kvcaware collectors — decoders + transports
hangangqiang
committed
[rollout] feat(router): kvcaware concrete strategy
hangangqiang
committed
[rollout] feat(router): kvcaware balancer e2e + config + generic example
hangangqiang
committed
[rollout] feat(router): split USE_VERL_STICKY into memory_overload_filter + slow_cut config fields
hangangqiang
committed
[rollout] feat(router): parallel_infer --device arg for Ascend backend
hangangqiang
committed
[feature] add inflight to s_load makesure select best replica in start phase

yyyyrf
authored and
hangangqiang
committed

Commits on Jul 23, 2026

Commits on Jul 24, 2026