BlazeFL is a lightweight framework for single-node Federated Learning (FL) simulation that eliminates the trade-off between throughput and reproducibility. By leveraging Python's free-threading architecture, BlazeFL allows for shared-memory execution that bypasses the heavy overhead of inter-process communication (IPC) and serialization.
Architecture overview: Shared-memory execution eliminates serialization/IPC overhead. Isolated RNG streams ensure deterministic results.
- 🚀 Zero-Copy Parallelism: Uses thread-based parallelism (PEP 703) to exchange model parameters via shared memory. Achieve up to 3.1x speedup on communication-heavy workloads compared to traditional process-based frameworks.
- 🔄 Bitwise Reproducibility: Guarantees identical results across runs, even with high concurrency. By assigning isolated RNG streams to each client, BlazeFL eliminates non-determinism caused by worker scheduling or completion-order-dependent aggregation.
- 🧩 No Framework Lock-in: Built on Python
Protocols. Integrate your existing PyTorch models and training loops with minimal changes. No rigid inheritance required. - 🍃 Minimalist Stack: Core execution relies only on PyTorch and Python standard libraries. Lightweight, easy to package, and highly portable.
BlazeFL keeps the simulation loop explicit and easy to debug. Here is the core pattern:
# Server-Client interaction is straightforward
while not handler.is_stopped():
# 1. Server: Sample clients and prepare downlink
sampled_clients = handler.sample_clients()
broadcast = handler.downlink_package()
# 2. Client Trainer: Process clients in parallel (Threads/Processes)
trainer.local_process(broadcast, sampled_clients)
uploads = trainer.uplink_package()
# 3. Server: Aggregate uploads
for pack in uploads:
handler.load(pack)BlazeFL offers three execution modes to suit your environment:
- Multi-Threaded (Recommended): Best for Python 3.14+ (Free-threading). Zero-copy sharing and minimal overhead.
- Multi-Process: Utilizes separate processes to achieve isolation. Uses shared-memory tensors for efficient parameter exchange.
- Single-Threaded: Simple, sequential execution. Perfect for debugging and initial prototyping.
Parallel execution often breaks reproducibility due to floating-point rounding differences and global random state interference. BlazeFL solves this with a Generator-Based Strategy:
- Each client is assigned a dedicated RNGSuite.
- Random operations (sampling, shuffling, augmentation) consume these client-isolated generators.
- Results are materialized in a deterministic order, ensuring bitwise-identical aggregation.
Tip
To ensure full determinism in vision pipelines, use transforms that accept explicit generators. BlazeFL provides utilities to snapshot and restore these states seamlessly.
In image classification experiments (CIFAR-10), BlazeFL substantially reduces execution time relative to widely used frameworks. The performance advantage is most pronounced in workloads where parameter exchange is the primary bottleneck.
uv add blazefl
# Or with reproducibility features
uv add blazefl[reproducibility]We welcome contributions! Please see our contribution guidelines.
@misc{azuma2026blazeflfastdeterministicfederated,
title={BlazeFL: Fast and Deterministic Federated Learning Simulation},
author={Kitsuya Azuma and Takayuki Nishio},
year={2026},
eprint={2604.03606},
archivePrefix={arXiv},
primaryClass={cs.LG},
url={https://arxiv.org/abs/2604.03606},
}