fix(render): two silent hangs in the turbo capture path - #132
Merged
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
andylizf
changed the base branch from
fix/page-height-viewport-bounded-body
to
main
July 31, 2026 08:44
The turbo path writes Chrome's raw BGRA tiles to a fixed
`/dev/shm/pixelrag_render/raw`. `/dev/shm` is shared by every user on the box,
so that directory belongs to whoever ran first — it lands at 0775 owned by them,
and every later user is denied write access to it.
Nothing about that surfaced. `mkdir(exist_ok=True)` still succeeds against the
existing directory; Chrome's write failure goes to its stderr, which this backend
sends to DEVNULL; and the resulting FileNotFoundError in every compression worker
was swallowed by a bare `except Exception: pass`. A second user on the same host
therefore got "Done: 13 tiles", a manifest listing 13 tiles with
`complete: true`, and an output directory holding nothing but `tiles.json`.
Scope the scratch path per uid, refuse to start when it isn't writable, and count
and log compression failures instead of dropping them.
Measured on a shared box, as the non-owning user:
before: Done: 13 tiles -> 0 images on disk
after: Done: 13 tiles -> 13 images on disk (2,611,261 bytes)
The pool is created from a thread while the capture side already runs its own
threads. Forking a multi-threaded process can leave the child holding a lock
that was never released, and the child then blocks forever — CPython has warned
about this since 3.12.
A single-threaded caller forks cleanly, which is why this survived: `pixelshot`
from a shell works every time. Under a threaded host it hangs with no output and
no error. The full test suite is one such host: `tests/test_render.py` passes in
5.8s on its own and hangs indefinitely when the async tests run first.
Switching to spawn also means the pool initializer has to survive pickling, so
the core-affinity helper moves from a closure to a module-level function.
before: full suite exceeded a 400s timeout, no result
after: 77 passed, 2 skipped in 12.17s
andylizf
force-pushed
the
fix/shm-scratch-per-user
branch
from
July 31, 2026 08:47
ef2c881 to
56e06de
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Two failures in the turbo capture path that both end the same way: the run reports
success and nothing is wrong in the logs.
1. The scratch directory belongs to whoever ran first
The turbo path writes Chrome's raw BGRA tiles to a fixed
/dev/shm/pixelrag_render/raw./dev/shmis shared by every user on the box, sothat directory belongs to whoever ran first — it lands at 0775 owned by them, and
every later user is denied write access to it.
Nothing about that surfaces:
mkdir(exist_ok=True)still succeeds against the existing directoryFileNotFoundErrorin every compression worker was swallowed by abare
except Exception: passA second user on the same host gets
Done: 13 tiles, a manifest listing 13 tileswith
complete: true, and an output directory holding nothing buttiles.json.Fix: scope the scratch path per uid, refuse to start when it isn't writable, and
count and log compression failures instead of dropping them.
Measured on a shared box, as the non-owning user:
2. The compression pool forked from a thread
fast_cdpbuilds its compression pool from inside a thread, while the capture sideis already running threads of its own. The pool used the platform default start
method,
fork— and forking a multi-threaded process can leave the child holding alock nobody will ever release, so it blocks forever. CPython has warned about this
since 3.12.
A single-threaded caller forks cleanly, which is why it survived:
pixelshotfrom ashell works every time. Under a threaded host it hangs with no output and no error.
The test suite is one such host:
tests/test_render.pyon its ownFix: start the pool with
spawn. That means the pool initializer has to survivepickling, so the core-affinity helper moves from a closure to a module-level function.
Based on #131 —
fast_cdp.pyhere already carries that PR's change.