You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
#3134 fixes the immediate user-visible failure mode of the memory-pressure ingress throttle (drain-mode feedback loop). The fix is surgical: skip the throttle when no module has work in flight. But the throttle is in the structurally wrong place, and the drain-mode bypass is a workaround for that.
The structural problem
The throttle lives in ScanIngress.get_incoming_event — the consumer side of the pipeline. That's backwards:
Queue-buffered memory pressure is caused by emit_event (producers adding to queues).
Queue-buffered memory pressure is relieved by get_incoming_event (the consumer removing from queues).
Backpressure belongs on the increaser, not the decreaser.
Throttling the consumer when queues are full means the only thing that frees memory is the only thing being slowed down. In the pathological case (drain-only mode, modules killed), this is a hard deadlock that requires the special-case bypass added in #3134.
Proposal
Move the throttle into BaseModule.emit_event. Every producer slows down proportionally under memory pressure; consumers stay unblocked and keep draining. Drain mode is self-handling: no producers running, nothing to throttle, ingress runs at full speed without needing a special-case predicate.
Why this is a real refactor, not a quick relocation
Every emit path needs to honor the delay: BaseModule.emit_event, _emit_event, intercept-module emit paths (ScanIngress.queue_event, ScanEgress.forward_event), output module emits.
Intercept modules emit too — throttling ScanIngress/ScanEgress reintroduces the drain-mode deadlock from the other end. They need to be exempt.
Async semantics differ: sleeping in emit_event blocks the caller's task, not a centralized consumer. Modules will compete for cycles differently than they do today.
Interaction with existing backpressure: per-module _qsize limits, the init_events seed-queue throttle (manager.py:67). Need to think through how the layers compose.
Context
#3134 fixes the immediate user-visible failure mode of the memory-pressure ingress throttle (drain-mode feedback loop). The fix is surgical: skip the throttle when no module has work in flight. But the throttle is in the structurally wrong place, and the drain-mode bypass is a workaround for that.
The structural problem
The throttle lives in
ScanIngress.get_incoming_event— the consumer side of the pipeline. That's backwards:emit_event(producers adding to queues).get_incoming_event(the consumer removing from queues).Throttling the consumer when queues are full means the only thing that frees memory is the only thing being slowed down. In the pathological case (drain-only mode, modules killed), this is a hard deadlock that requires the special-case bypass added in #3134.
Proposal
Move the throttle into
BaseModule.emit_event. Every producer slows down proportionally under memory pressure; consumers stay unblocked and keep draining. Drain mode is self-handling: no producers running, nothing to throttle, ingress runs at full speed without needing a special-case predicate.Why this is a real refactor, not a quick relocation
BaseModule.emit_event,_emit_event, intercept-module emit paths (ScanIngress.queue_event,ScanEgress.forward_event), output module emits.ScanIngress/ScanEgressreintroduces the drain-mode deadlock from the other end. They need to be exempt.emit_eventblocks the caller's task, not a centralized consumer. Modules will compete for cycles differently than they do today._qsizelimits, theinit_eventsseed-queue throttle (manager.py:67). Need to think through how the layers compose.References
bbot/scanner/manager.py:147-157(the sleep)bbot/scanner/scanner.py:755-768(status loop)bbot/scanner/scanner.py:841-848694ea643c(PR Scan memory benchmarks: RSS sampling, mid-scan census, parallel-chains workload #3083)