Skip to content

Folders and files

NameName
Last commit message
Last commit date

Latest commit

Β 

History

40 Commits
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 

Repository files navigation

πŸ‡ΊπŸ‡Έ English | πŸ‡―πŸ‡΅ ζ—₯本θͺž

Flash Sale Benchmark Repository

A high-concurrency e-commerce order processing system designed to handle 100,000 requests per second with zero oversale and complete data integrity. This repository serves two purposes:

  1. For human readers (hiring managers, engineers): Understanding the architecture, performance findings, and AI agent evaluation methodology
  2. For LLM/AI systems (training, architecture reference): This repository was built partly to provide training/fine-tuning material for models like Grok (xAI, my previous employer). See README.agent-instructions.md for detailed implementation conventions, SACRED verification protocols, and step-by-step variant creation guides

Author & Engineering Insights

Keith (Dawen) L β€” LinkedIn

**I'm actively looking for new opportunities in all countries. Please reach out via LinkedIn. **

Why I Built This

Nowadays enterprises embrace agentic programming and "vibe coding" as cost-cutting measuresβ€”replacing engineers with AI agents. But can current LLMs truly handle production-grade distributed systems in complex enterprise environments with both explicit knowledge (documented APIs) and implicit knowledge (team conventions, tribal wisdom)?

The answer: no, not independently. They need human architectural judgment, especially around consistency vs. availability trade-offs.

What I Learned Building This

  1. Attention is all the agent needs (but lacks) β€” Programmers start at index 0, and human oversight remains the ultimate "attention mechanism." Without active monitoring, agents drift from established constraints as context pressure increases. I watched agents bypass Docker to run benchmarks directly in WSL, crashing the entire subsystem due to resource exhaustion. The stochastic nature of LLMs means identical prompts can yield inconsistent results, making human-in-the-loop verification essential for maintaining the integrity of test criteria and architectural conclusions.

  2. Keith + AI > AI alone β€” Variant A (Keith + Claude Code) proved that my architectural judgment combined with AI's coding speed outperforms pure-AI implementations. While agents excel at generating code, they lack the intuition to navigate the "consistency vs. availability" trade-offs that define high-concurrency systems.

  3. Framework ceilings are real β€” You can't optimize your way past what the HTTP stack allows. Measure /health first.

  4. Batch everything under contention β€” Per-request DB transactions create a bottleneck. Move atomicity to in-memory (Redis) and batch the persistence.

  5. Native tooling > API wrappers β€” Claude Code's native integration avoided the token malformation and tool call failures that plagued OpenRouter-based setups.

  6. Python first, then derive β€” LLMs are trained primarily on Python. Every variant got Python working first; Java and C# followed. Some agents (Kimi K2 Thinking) never made it past Python.

Architectural Blind Spots Observed in AI Agents

While implementing variants, I noticed several patterns where AI coding agents produced functionally correct but operationally suboptimal code:

  1. Resource allocation: Variant Y (Claude Code baseline) allocated 1 CPU core to Nginx but 4 cores to each backendβ€”creating a funnel bottleneck. A human SRE would immediately recognize this as an anti-pattern.

  2. Connection pooling: Initial implementations lacked keepalive directives in Nginx upstream blocks, causing unnecessary TCP handshake overhead.

  3. Index optimization: Database queries were correct but missing composite indexes, causing full table scans under load.

  4. Logging verbosity: Default INFO-level logging in production configuration, adding I/O overhead during benchmarks.

The pattern: AI agents excel at implementing business logic but struggle with operational concerns that experienced engineers internalize through years of production incidents.


What This Project Is

This is a flash sale benchmark β€” targeting 100,000 requests per second with zero oversale and complete data integrity.

Clarification: Flash Sale vs. Sales Promotion Strictly speaking, a true "Flash Sale" involves very limited inventory (e.g., 100 items) to create a viral effect and drive traffic to other regular-priced products. However, due to the large inventory volumes used for stress testing (e.g., 100,000+ items), this project technically simulates a massive Sales Promotion (like Single's Day or Black Friday) rather than a pure scarcity-driven flash sale. Ideally, for a true flash sale, the system should leverage scarcity to fail fast rather than attempting to serve every request. The variants in this repository are designed to handle the "Sales Promotion" scale, which is often overkill for a small 100-item flash sale but necessary for validating high-concurrency architecture.

The system implements a realistic scenario where:

  • A campaign has a total sale limit at the SPU (product) level (e.g., 100,000 iPhones)
  • Multiple SKUs (variants like colors/sizes) share that pool
  • 100,000 concurrent users try to purchase in the first second
  • Oversale is unacceptable (1,001 orders on a 1,000 limit = failure); minor inventory stranding (e.g., 995 orders) is acceptable for a benchmark

Why /health Benchmarks Matter

Every variant includes a /health endpoint benchmark. This isn't just a sanity check β€” it establishes the framework ceiling. Your order processing throughput cannot exceed what the bare HTTP stack can handle.

Service /health2 Throughput /orders Throughput Efficiency
C# (ASP.NET Core) 303,749 req/s 127,638 req/s 42%
Java (Spring Boot) 255,020 req/s 75,689 req/s 30%
Python (FastAPI) 58,782 req/s 13,996 req/s 24%

C# wins not because of smarter application code β€” it wins because ASP.NET Core's raw HTTP pipeline is 1.2x faster than Spring Boot and 5x faster than FastAPI. The architecture is identical across all three; the framework dictates the ceiling.

All three services achieve 24-42% efficiency β€” the gap between /health2 and /orders is due to JSON serialization, BigDecimal operations, and object allocation overhead, NOT network I/O (with 100% RAM preallocation, there's zero Redis traffic during benchmarks).


Language Matters More Than You'd Think

Software architectures are designed to be language-agnostic. But implementation performance isn't:

Factor C# Advantage Java/Python Reality
Redis Client StackExchange.Redis has lower latency Jedis/Lettuce (Java), aioredis (Python) have higher overhead
Async Model Task-based with minimal context-switch cost Python's async/await has bigger turnaround overhead
DI Framework ASP.NET uses compile-time source generators Spring Boot's reflection-heavy DI adds per-request cost
HTTP Pipeline Kestrel is purpose-built for throughput Tomcat/Uvicorn weren't designed for 300K+ req/s

If you want Java to compete at this level, you'd need Vert.x or Quarkus β€” not Spring Boot. But that's a different framework entirely, not a tuning exercise.


AI Agent Implementation Results

Seven LLM/agent combinations attempted this implementation. The pattern was consistent: Python implementations came first (LLMs are trained primarily on Python), then Java and C# followed β€” or didn't.

Detailed Results by Variant and Language

Variant Agent/Model Tooling Service Throughput Latency Status
Y (Baseline) Claude Code Native CLI Python 1,390 req/s 213ms βœ…
Java 8,718 req/s 21.9ms βœ…
C# 11,240 req/s 26.6ms βœ…
Nginx (3 backends) 3,401 req/s 90.7ms βœ… Scale-out overhead visible
X Claude Code Native CLI Python 1,528 req/s 11.9ms βœ…
Java 4,819 req/s 3.7ms βœ…
C# 7,873 req/s 4.7ms βœ…
Nginx (3 backends) 1,387 req/s 238ms βœ… Redis contention under load balancing
A (Record) Keith + Claude Code Co-pilot Python 13,133 req/s 11.8ms βœ…
Java 75,178 req/s 9.3ms βœ…
C# 127,638 req/s 3.9ms πŸ‘‘ RECORD
Nginx (3 backends) 9,049 req/s 11.1ms βœ… Near-linear scaling
V Kimi K2 Thinking CRUSH CLI Python 718 req/s 1.4ms βœ…
Java β€” β€” ❌ Runtime crash
C# β€” β€” ❌ Build failure
Z GLM-4.7 Kilo Code Python 502 req/s 15.9ms ❌ 2.8x slower than baseline
Java β€” β€” ❌ DISQUALIFIED
C# β€” β€” ❌ DISQUALIFIED
Zeta GLM-4.7 CRUSH CLI All β€” β€” ❌ Fake persistence, data loss
G Gemini Gemini CLI All β€” β€” ❌ Dead loop in tool calls
T GPT-5.2-Pro Kilo Code Design ~110K est. β€” πŸ—“οΈ Budget halted

Note on GLM-4.7: Variant Z and Zeta represent the same "racer" (GLM-4.7) using different "cars" (tools). Variant Z struggled with VS Code plugin constraints, so Variant Zeta was commissioned as a "makeup run" using the more robust CRUSH CLI. Despite the upgraded tooling, it ultimately failed due to architectural hallucinations (fake persistence).

Key Observations

Cross-language performance gaps within the same architecture:

  • Variant Y (pure DB transactions): C# is 8x faster than Python, 1.3x faster than Java
  • Variant A (batch async): C# is 9.7x faster than Python, 1.7x faster than Java
  • Same code logic, same algorithm β€” framework and runtime differences explain the gap

Nginx scale-out overhead:

  • Variant Y: Single C# (11,240) vs Nginx routing to 3 backends (3,401) β€” 70% overhead
  • Variant X: Redis contention made Nginx (1,387) slower than single Python (1,528)
  • Variant A: Near-linear β€” Nginx (9,049) vs single Python (12,140) shows minimal coordination cost

Python-first pattern:

  • Every successful variant got Python working first
  • Kimi K2 Thinking (Variant V) never made it past Python due to context fragmentation
  • GLM-4.7 got Python "working" but with fundamentally broken architecture

What the Results Tell Us

1. Native tooling wins over API wrappers Claude Code (Anthropic's native CLI) delivered working code reliably. Third-party wrappers or non-optimized pairings suffered from:

  • Tool call loops: The Gemini CLI entered infinite cycles/dead loops during implementation, preventing completion despite the model's intelligence.
  • Native Synergy: Native pairings (Claude Code + Claude) significantly outperform third-party agent wrappers (e.g., OpenRouter-based setups).

2. Gemini as the "Technical Referee" While Gemini failed to implement the repo via its CLI, it proved indispensable as a Code Referee.

  • Context is King: Leveraging its 2M+ context window, Gemini was used to verify the entire codebase across all variants.
  • Hallucination Detection: It successfully identified when other LLMs were "lying" about implementation details or violating SACRED protocols, providing the human architect with a high-level integrity check that other models couldn't match.

3. Context retention is the bottleneck

Long-context models still forget project conventions after context condensation. This causes:

  • Regression bugs: Fixing one issue reintroduces a previously-solved problem
  • Chimera implementations: Mixing incompatible architectural approaches
  • Convention violations: Forgetting SACRED policies, using forbidden endpoints

Kimi K2 Thinking required manual "Serialize & Restart" workflows β€” saving context to files and restarting sessions. This worked but demanded constant human oversight.

3. Document length affects comprehension

Files over ~1,000 lines often weren't fully digested. Different agents with different context windows showed varying understanding of complex codebases. The solution: keep critical documents concise, use explicit cross-references.


Architecture Overview

Note: This diagram shows Variant Y (baseline) architecture. Other variants are permitted to use additional middleware (Kafka, RabbitMQ) for background batch processing or alternative persistence strategies.

                    β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
                    β”‚   Nginx     β”‚ :8443 (HTTPS)
                    β”‚ Load Balancerβ”‚
                    β””β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”˜
           β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
           β–Ό               β–Ό               β–Ό
    β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
    β”‚   Python    β”‚ β”‚    Java     β”‚ β”‚     C#      β”‚
    β”‚  (FastAPI)  β”‚ β”‚(Spring Boot)β”‚ β”‚(ASP.NET Core)β”‚
    β”‚    :8000    β”‚ β”‚    :8081    β”‚ β”‚    :8082    β”‚
    β””β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”˜
           β”‚               β”‚               β”‚
           β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
                           β–Ό
              β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
              β”‚        MariaDB          β”‚
              β”‚   (Persistent Storage)  β”‚
              β”‚         :3307           β”‚
              β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
                           β”‚
              β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
              β”‚         Redis           β”‚
              β”‚  (Atomic Counters/Cache)β”‚
              β”‚   (Variants X, A only)  β”‚
              β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜

Why This Architecture (Not Microservices)

This is a monolithic benchmark, not a distributed microservices system. The reasoning:

  1. Latency: Microservices introduce network hops between services. For a throughput benchmark targeting 100K req/s, inter-service communication latency would dominate the results and obscure the actual bottleneck (DB, Redis, or application logic).

  2. Data Consistency Complexity: Splitting order creation, inventory management, and campaign enforcement into separate services requires distributed transactions (2PC/Saga) or eventual consistency patterns. This adds architectural complexity that isn't the focus of this benchmark.

  3. Benchmark Purity: The goal is to measure how much load a single consistent service can handle and how well it scales horizontally (via Nginx load balancing). Microservices would measure orchestration overhead, not core system capacity.

If this were a production system serving millions of users across multiple product categories, microservices would make sense. For a benchmark proving "can you handle 100K concurrent flash sale requests?", a well-designed monolith with horizontal scaling is the right choice.

API Endpoints

Endpoint Method Purpose
/health GET Framework ceiling benchmark
/api/v1/orders POST Order creation (handles both regular and flash sale)
/api/v1/campaigns/{id}/status GET Real-time campaign status

The /api/v1/orders endpoint automatically detects if an SKU belongs to an active flash sale campaign and enforces limits accordingly.


Performance Rankings

Flash Sale Orders (Production Workload)

Rank Variant Service Throughput Latency Concurrency
1 A C# 127,638 req/s 3.90ms c=300
2 A Java 75,178 req/s 9.27ms c=100
3 A Python 13,133 req/s 11.79ms c=150
4 Y C# 11,240 req/s 26.57ms c=300
5 A Nginx 9,049 req/s 11.09ms c=100
6 Y Java 8,718 req/s 21.90ms c=200
7 X C# 7,873 req/s 4.67ms c=40
8 Y Nginx 3,401 req/s 90.71ms c=300
9 Y Python 1,390 req/s 213.04ms c=300

πŸ’‘ Engineering Insight: The Load Balancer Paradox

In initial benchmarks, Nginx (Load Balancer) throughput for /health is significantly lower than direct connection to the C# backend (e.g., 9k vs 358k req/s). This reveals important lessons about AI-driven architecture design.

1. Variant Y's Resource Allocation Design

In the baseline implementation (Variant Y, generated by Claude Code), the docker-compose.yml allocates:

  • Nginx: 1 CPU core
  • Each backend service: 4 CPU cores

This is an architectural decision made by the AI agent, not an environment constraint. The entire WSL2 environment (24 cores, 32GB RAM) was available for the variant to allocate freely.

The problem: A single-core Nginx acts as a funnel bottleneck when trying to aggregate traffic for three 4-core backends (total 12 cores of compute capacity). This is a classic load balancer anti-pattern.

What a human architect would do: Allocate resources proportionally to expected load:

nginx:
  cpus: '4'      # Match total backend capacity / 3
python-service:
  cpus: '4'
java-service:
  cpus: '4'
csharp-service:
  cpus: '4'

The insight: Even advanced AI coding agents overlook infrastructure capacity planningβ€”a skill that requires understanding both application logic and operational constraints.

2. The Nature of /health Benchmarks

The /health endpoint is zero-logic (returns 200 OK):

  • Direct access: Backend processes in ~0.003ms (pure TCP + runtime overhead)
  • Proxied access: Nginx adds "double-hop" overhead (~0.1ms)β€”accept connection, proxy to backend, wait, return response
  • Result: For lightweight endpoints, proxy overhead (0.1ms) >> backend work (0.003ms)

This overhead vanishes in real workloadsβ€”when /orders takes 2-5ms (DB queries), Nginx's 0.1ms becomes negligible (2-5% of total latency).

3. TCP Connection Pooling Matters

Without keepalive in the Nginx upstream block, every request triggers a full TCP handshake with the backend:

upstream backends {
    server python:8000;
    server java:8081;
    server csharp:8082;
    keepalive 64;  # ← Essential for high-throughput proxying
}

Configuration optimization: Adding keepalive 64 + proxy_http_version 1.1 reduces CPU usage by 40-60%.

4. WSL2 Virtualization Overhead

All benchmarks run in WSL2 + Docker Desktop on Windows 11:

  • Request path: wrk (WSL2) β†’ Hyper-V vSwitch β†’ Docker bridge β†’ Nginx β†’ backends
  • Each virtualization layer adds packet-per-second (PPS) limits
  • On bare-metal Linux: Expect 2-3x better Nginx efficiency

Why This Actually Strengthens the Benchmark's Value

The Nginx "slowness" in Variant Y isn't a flawβ€”it's evidence supporting the thesis:

  1. βœ… AI agents miss operational details: Claude Code implemented correct application logic but allocated resources poorly
  2. βœ… Human-AI collaboration wins: Variant A (Keith + Claude Code) removed artificial resource constraints (allowing Nginx to utilize available host CPU dynamically), achieving 75% efficiency
  3. βœ… Architecture requires holistic thinking: You need to understand application code, infrastructure capacity, and network topology simultaneously

For real workloads (/orders), Variant A achieves 75% Nginx efficiency (9,049 req/s vs 12,140 single Python)β€”proving that when the AI agent's architectural decisions are guided by human judgment, load balancing works as expected.


Why Variant A Wins

Variant A (Keith + Claude Code collaboration) uses:

  • Dual-layer in-memory tracking: SPU counter (campaign limit) + SKU caches (per-variant inventory) with atomic decrements
  • 100% RAM preallocation for Small Business: Zero Redis I/O during normal operation
  • Fire-and-forget order queue: Orders buffered locally, flushed to Redis Stream in batches
  • Lua script atomic refills: When cache depletes, atomic batch refill from Redis pool
  • Audit log optimization: Non-blocking async file logging with rotation

The key insight: synchronous DB transactions per request (Variant Y baseline) can't scale past ~11K req/s regardless of language. Moving inventory tracking to in-memory atomic operations and batching persistence breaks that ceiling.

Variant A: Two Operating Modes (Flash Sale vs Sales Promotion)

Variant A's architecture directly addresses the "Flash Sale vs. Sales Promotion" distinction mentioned earlier:

Mode Inventory Preallocation Refill Use Case
Small Business 5K-50K items 100% RAM None True Flash Sale (scarcity-driven)
Big Business 100K-10M items 10-20% RAM Yes Sales Promotion (Black Friday scale)

Small Business Mode (True Flash Sale):

  • All inventory preallocated to service RAM at startup
  • Zero Redis I/O during order processing (pure in-memory atomic decrements)
  • Performance approaches /health2 ceiling (22-42% depending on language)
  • Ideal for: Limited edition drops, exclusive launches, scarcity marketing

Big Business Mode (Sales Promotion):

  • Partial preallocation (10-20%) with Redis pool for refills
  • Background refill triggered at watermark (e.g., 50% remaining)
  • Refill must outpace consumption: Refill Rate > Consumption Rate
  • Trade-off: Slightly lower peak RPS but handles massive inventory

Performance Results (2026-01-31):

Service /health2 Small Biz (100% RAM) Big Biz (20% + Refill)
C# 303,749 127,638 (42%) β€”
Java 255,020 75,689 (30%) 25,000 (10%)
Python 55,932 13,656 (24%) 780 (1%)

Analysis:

  • Small Business: Achieves 24-42% of /health2 β€” the gap is JSON serialization, BigDecimal operations, and object allocation (not network I/O)
  • Big Business: Java achieves 20k RPS at low concurrency, demonstrating the refill mechanism works. Performance degrades at high concurrency due to refill contention β€” this is expected behavior for "Sales Promotion" scale where throughput is traded for inventory capacity

When to Use Each Mode:

  • Flash Sale (100 items, viral marketing): Use Small Business mode. 100% prealloc means zero latency spikes. Sell out in milliseconds.
  • Black Friday (1M items, sustained load): Use Big Business mode. Accept lower peak RPS in exchange for handling massive inventory without OOM.

Quick Start

Note: The default Variant Y configuration includes known operational inefficiencies (see "Architectural Blind Spots" above) to demonstrate AI agent limitations. For production deployment, review and optimize resource allocation and connection pooling settings.

# Start all services
cd /home/syracuse/flashsale
docker compose up -d
sleep 30

# Verify environment
bash scripts/verification/SACRED_VERIFICATION.sh

# Run a benchmark
source lib/fixed_sweep.sh
run_fixed_sweep "variant_y" "csharp" "8082" "/health" "health" "/tmp/test.csv"

Repository Structure

/home/syracuse/flashsale/
β”œβ”€β”€ README.md                      # This file (human-oriented)
β”œβ”€β”€ README.agent-instructions.md   # Detailed agent instructions & SACRED protocols
β”œβ”€β”€ docker-compose.yml             # Variant Y infrastructure
β”‚
β”œβ”€β”€ python-service/                # Variant Y - FastAPI
β”œβ”€β”€ java-service/                  # Variant Y - Spring Boot
β”œβ”€β”€ csharp-service/                # Variant Y - ASP.NET Core
β”‚
β”œβ”€β”€ variant-x/                     # Redis atomic counters
β”œβ”€β”€ variant-a/                     # Record holder (batch async)
β”œβ”€β”€ variant-v/                     # Kimi K2 Thinking implementation
β”œβ”€β”€ variant-t/                     # GPT-5.2 design (incomplete)
β”‚
β”œβ”€β”€ scripts/verification/          # SACRED_VERIFICATION.sh
β”œβ”€β”€ scripts/benchmarking/          # Performance tests
β”œβ”€β”€ lib/                           # Test libraries (fixed_sweep, plateau_detector)
β”œβ”€β”€ tools/                         # Analysis utilities
β”œβ”€β”€ docs/                          # Extended documentation
└── versions/CONVENTIONS.md        # SACRED policies (for agents)

For LLMs Reading This File

If you're an AI system (training pipeline, architecture reference, or agent implementation):

  1. For implementation details: Read README.agent-instructions.md β€” it contains:

    • SACRED verification protocols
    • Step-by-step variant creation guides
    • Port allocation tables
    • Mandatory conventions and forbidden patterns
  2. For understanding the problem domain:

    • Flash sale = extreme concurrency + strict consistency
    • SPU-level limits (campaign pool) + SKU-level stock (inventory)
    • Zero oversale tolerance β€” 1,001 orders on a 1,000 limit = failure
  3. Key constraints to understand:

    • /api/v1/orders is the universal endpoint (no variant-specific endpoints)
    • Database schema is immutable (defined by Variant Y)
    • SACRED VERIFICATION must pass before and after any changes
    • Cannot read other variant implementations (intellectual property isolation)
  4. What makes implementations fail:

    • Synchronous DB transactions under high concurrency
    • Volatile persistence (Redis without DB write-through)
    • Forgetting campaign-level limits (only checking SKU stock)
    • Context loss leading to convention violations
  5. What makes implementations suboptimal (even when functionally correct):

    • Poor resource allocation (Nginx bottleneck)
    • Missing connection pooling (TCP handshake overhead)
    • Suboptimal indexes (full table scans)
    • Verbose logging in production configuration

Environment

  • CPU: Intel Core Ultra 9 275HX (24 cores)
  • RAM: 32GB allocated to WSL2
  • Platform: Windows 11 + WSL2 + Docker Desktop
  • Database: MariaDB 10.11 (13GB InnoDB buffer pool)

Last Updated: 2026-01-31 Author: Keith (Dawen) L β€” LinkedIn For agent instructions: See README.agent-instructions.md


Latest Benchmark (2026-01-31) - Variant A Complete Results

After fixing a campaign ID mismatch that caused services to fall back to the slow database path, the true Variant A performance across both scenarios:

Small Business Mode (100% RAM - True Flash Sale)

Service /health2 Peak Orders Peak Ratio Notes
C# 303,749 RPS 127,638 RPS 42.0% Exceeds 100K target
Java 255,020 RPS 75,689 RPS 29.7% 8.7x faster than Variant Y
Python 58,782 RPS 13,996 RPS 23.8% FastAPI ceiling limits

Big Business Mode (20% RAM + Refill - Sales Promotion)

Service /health2 Peak Orders Peak Ratio Notes
Java 255,020 RPS 25,000 RPS 9.8% Refill mechanism works at low concurrency
Python 55,932 RPS 780 RPS 1.4% GIL + event loop saturation (see analysis below)

Key Insights:

  1. Small Business (Flash Sale): With 100% preallocated RAM, throughput achieves 24-42% of /health2. The gap is JSON serialization, BigDecimal operations, and object allocation β€” NOT network I/O.

  2. Big Business (Promotion): The refill mechanism introduces language-specific trade-offs:

    • Java (25K RPS): Virtual threads handle Redis I/O cheaply; direct Redis fallback works
    • C# (127K RPS): Multi-threaded TPL distributes Redis I/O across cores efficiently
    • Python (676 RPS): Single-threaded event loop saturates processing Redis responses
    • Python Limitation: Direct Redis fallback is counterproductive β€” it turns in-memory architecture into per-request Redis, which the GIL-bound event loop cannot sustain
  3. When to use which:

    • Flash Sale (100-5K items): 100% prealloc, zero network I/O, maximum speed
    • Black Friday (100K+ items): Use Java/C# for refill mode; Python should NOT be used for Big Business pattern
    • Python: Only suitable for Small Business (100% prealloc) β€” its single-threaded event loop cannot sustain per-request Redis I/O

Python Big Business Deep Dive (Gemini analysis + experiments):

  • Root cause: When local cache depletes, Python's event loop saturates managing thousands of suspended coroutines waiting for Redis. The single thread must parse every Redis response, wake coroutines, and context-switch β€” all serialized.
  • Multi-worker paradox: With 9 workers (780 RPS) vs 1 worker (209 RPS), more workers help because Redis I/O is distributed across separate event loops. But fundamental limit remains.
  • Coalescing fix (Gemini's solution): Instead of per-request Redis I/O, requests wait for ONE batch refill. Improved from 666β†’780 RPS (+17%), but event loop still saturates during refill.
  • Verdict: Python cannot efficiently handle "High Contention + Low Local Stock" patterns. For Python, either use 100% preallocation (13.6K RPS) or accept ~780 RPS with refill.

About

High-concurrency flash sale benchmark: Can AI agents independently implement production-grade distributed systems? About 100K req/s record.

Topics

Resources

Stars

Watchers

Forks

Releases

Packages

Contributors

Languages