An imperative command-line-interface for AI workload orchestration

A lightweight LLM inference runtime, evolving toward heterogeneous, stage-disaggregated serving.

A policy layer above transport for KV movement, workload-aware admissibility, and explainable routing in disaggregated inference.

Open-source Python control plane for LLM inference: launch, route, scale, and observe disaggregated serving on infrastructure you control.

DiSpec — a from-scratch LLM inference engine: paged attention, continuous batching, CUDA-graph decode, speculative decoding, and prefill/decode disaggregation

Research fork of LLMServingSim 2.0 investigating Head-of-Line (HoL) blocking mitigation in disaggregated LLM serving via novel scheduling algorithms.

A fault-tolerant LLM routing system that decouples inference from AWS Bedrock by routing prefill and decode tasks through SQS and ensuring zero-downtime scaling with a graceful drain sidecar.