Skip to content

Commit ef2c881

Browse files
committed
fix(render): start the compression pool with spawn, not fork
The pool is created from a thread while the capture side already runs its own threads. Forking a multi-threaded process can leave the child holding a lock that was never released, and the child then blocks forever — CPython has warned about this since 3.12. A single-threaded caller forks cleanly, which is why this survived: `pixelshot` from a shell works every time. Under a threaded host it hangs with no output and no error. The full test suite is one such host: `tests/test_render.py` passes in 5.8s on its own and hangs indefinitely when the async tests run first. Switching to spawn also means the pool initializer has to survive pickling, so the core-affinity helper moves from a closure to a module-level function. before: full suite exceeded a 400s timeout, no result after: 77 passed, 2 skipped in 12.17s
1 parent 8e4e747 commit ef2c881

1 file changed

Lines changed: 26 additions & 9 deletions

File tree

render/src/pixelrag_render/backends/fast_cdp.py

Lines changed: 26 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -99,6 +99,18 @@
9999
# ---------------------------------------------------------------------------
100100

101101

102+
def _pin_to_cores(cores) -> None:
103+
"""Pool initializer: keep compression workers off the capture cores.
104+
105+
Module level, not a closure, because the pool is started with "spawn" — the
106+
initializer has to survive pickling, which a nested function does not.
107+
"""
108+
try:
109+
os.sched_setaffinity(0, cores)
110+
except (OSError, AttributeError):
111+
pass # not Linux, or the affinity call is unavailable — harmless
112+
113+
102114
def _raw_scratch_dir() -> Path:
103115
"""Per-user scratch directory on /dev/shm for Chrome's raw BGRA dumps.
104116
@@ -321,7 +333,7 @@ async def _run_render(
321333
# The thread runs pool.starmap in batches, fully independent of asyncio.
322334
import queue as _queue
323335
import threading
324-
from multiprocessing import Pool as MPPool
336+
import multiprocessing
325337

326338
metrics = {
327339
"total_tiles": 0,
@@ -335,15 +347,20 @@ async def _run_render(
335347
n_cpus = os.cpu_count() or 128
336348
compress_cores = set(range(max(0, n_cpus - n_compressors), n_cpus))
337349

338-
def _pool_init():
339-
try:
340-
os.sched_setaffinity(0, compress_cores)
341-
except OSError:
342-
pass
343-
344350
def _compressor_thread():
345-
pool = MPPool(processes=n_compressors, initializer=_pool_init)
346-
# Warm up: ensure all workers are forked and idle before capture starts
351+
# "spawn", not the platform default. This pool is created from a thread while
352+
# the capture side already runs its own threads, and forking a multi-threaded
353+
# process can deadlock the child on a lock that was held at fork time — CPython
354+
# warns about exactly this since 3.12. In a single-threaded caller the fork
355+
# happens to work, which is why it survived so long; under any threaded host
356+
# (a test session that ran async tests first, a web server calling into the
357+
# renderer) it hangs with no output and no error.
358+
pool = multiprocessing.get_context("spawn").Pool(
359+
processes=n_compressors,
360+
initializer=_pin_to_cores,
361+
initargs=(compress_cores,),
362+
)
363+
# Warm up: ensure all workers are up and idle before capture starts
347364
pool.map(int, range(n_compressors))
348365
async_results = []
349366
while True:

0 commit comments

Comments
 (0)