Skip to content
Open
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
130 changes: 130 additions & 0 deletions docs/decisions/17674-network-original-body-and-timing.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,130 @@
# 17674. Network responses expose their original body and request timing

- Status: Proposed
- Date: 2026-06-11
- Discussion: https://github.com/SeleniumHQ/selenium/pull/17674

## Context

The high-level network handlers let users mutate or stub requests and responses, but they
cannot **read the original response body**. The two most common monitoring needs are
therefore unmet:

1. **Read the body** of a response (to assert on an API payload, or to fetch-then-patch:
read the real body, edit one field, return the edited version).
2. **Read request timing** β€” DNS, connect, TLS, request, response phases β€” for
performance assertions.

Both are now supported by the BiDi protocol. `network.addDataCollector` registers a
collector for a data type (`response`, and increasingly `request`) with a
`maxEncodedDataSize` cap; `network.getData` returns the captured `bytes` for a request id;
`network.removeDataCollector`/`disownData` manage lifecycle. Timing already arrives on
every request: `network.RequestData.timings` is a `FetchTimingInfo` with thirteen fields
(`timeOrigin`, `requestTime`, `redirectStart/End`, `fetchStart`, `dnsStart/End`,
`connectStart/End`, `tlsStart`, `requestStart`, `responseStart`, `responseEnd`).

The constraint: data collectors are **newer and unevenly implemented**. Firefox supports
the `response` type (with `maxEncodedDataSize` required) and recently added `request`;
Chromium supports it (Puppeteer consumes it) but coverage and quirks differ (e.g. known
Firefox redirect-timing edge cases). So body-read must degrade gracefully where a browser
lacks support, rather than hard-failing. Timing, by contrast, is plain event data with no
collector dependency.

Playwright exposes `response.body()/text()/json()` and `route.fetch()` (fetch-then-patch),
plus a `request.timing` dict β€” the shape users expect.

## Decision

Bindings expose, on their existing `Response`/`Request` wrappers:

- **`body()` / `text()` / `json()`** on a response, reading the **original** payload via a

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This ADR should not require text() or json() helpers at this time. If we are intentionally deferring larger functional concerns like observation-vs-interception, we should also defer these kinds of helper methods.

The cross-binding contract should be limited to exposing the collected body in a binary-safe form. Text decoding and JSON parsing would add an unnecessary surface area around content-type parsing, charset handling, invalid encodings, binary payloads, compressed bodies, and language-specific JSON return types that are not a good priority.

Users can do these conversions themselves with the libraries and assumptions appropriate for their application. We can add convenience helpers later if there is clear demand.

data collector (`addDataCollector` + `getData`). The binding manages collector
lifecycle; users do not handle collector ids for the common case. Where the browser does
not support data collectors, these methods raise a **clear, typed "unsupported"
error** β€” never a silent empty body. Bindings SHOULD expose a capability check so users

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Selenium should not proactively couple itself to transient browser implementation matrices. Selenium does need an error handling policy (see point 4 in #17685), but it should be passing through what the browser responds with rather than requiring temporary gating logic.

can branch.
- **Fetch-then-patch**: a response handler can read the original body, modify it, and

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This can work for requests, but not for responses as the spec is currently written.

network.responseStarted is emitted after response headers are received but before the body is complete. If the event is blocked, the remote end stores the request in the blocked request map and then awaits "continue request". The response cannot reach responseCompleted while responseStarted remains blocked by the interception, so the callable can never access the body.

Do we currently have a way to differentiate add_response_handler for whether the phase is started or complete? Do we need separate methods for this?

supply the modified version (mapping to `provideResponse`, carrying over status and
headers). This reuses the body-read mechanism above.
- **`timing`** on a request, exposing the `FetchTimingInfo` fields through named
accessors. Timing is always available (no collector), so it never raises "unsupported".
- Convenience header accessors consistent with the wrappers: `header_value(name)` and an
all-headers accessor, since collectors and timing make richer introspection common.

Code sketch β€” Python (reference implementation):

```python
def handler(response):
if response.body_supported(): # capability gate
data = response.json() # original body via addDataCollector + getData
print(response.status, data["items"])
t = response.request.timing # always available
print(t.response_end - t.request_time) # total ms

driver.network.add_response_handler("**/api/**", handler)

# Fetch-then-patch: read original, edit, return edited
def patch(response):
body = response.json()
body["feature_flag"] = True
response.provide(json=body) # -> provideResponse, status/headers carried over

driver.network.add_response_handler("**/api/config", patch)
```

Code sketch β€” other bindings (idiomatic shape, same semantics):

```java
driver.network().addResponseHandler("**/api/**", response -> {
if (response.bodySupported()) {
String text = response.text(); // original body
}
FetchTiming t = response.request().timing();
});
```

## Considered options

- **Body-read via managed data collectors, with a typed unsupported error + capability
gate; always-on timing (chosen)** β€” gives the Playwright-shaped API where the browser
supports it and fails honestly where it does not; timing needs no gate.
- **Wait for universal browser support before exposing body-read** β€” avoids the
degradation path, but withholds a high-value, already-shipping feature from users on
browsers that do support it (Firefox, Chromium). Rejected: gate, don't withhold.
- **Expose raw `add_data_collector`/`get_data` only** β€” already generated, but pushes
collector-id and lifecycle management onto every user and offers no `body()`/`json()`
ergonomics. Rejected as the user-facing answer; the raw commands remain as an escape
hatch.
- **Silently return an empty body when unsupported** β€” superficially simpler, but turns a
capability gap into a silent wrong result. Rejected outright.

## Consequences

- Users can assert on and patch real response payloads, and assert on request timing, in a
consistent shape across bindings.
- Body-read carries a **browser-support matrix** the binding must track and document;
tests for it gate on capability. Collector `maxEncodedDataSize` caps mean very large
bodies may be truncated β€” bindings choose and document a default cap.
- Bindings own collector lifecycle (add on demand, remove/disown appropriately) so users
do not leak collectors.
- Timing exposure is cheap and unconditional; it can land ahead of body-read.

## Binding status

| Binding | Status | Notes / tracking link |
|------------|---------|----------------------------------------------------------------------|
| Java | pending | |
| Python | pending | request/response handler wrappers exist; body-read + timing not yet built; raw `add_data_collector`/`get_data` generated |
| Ruby | pending | |
| .NET | pending | |
| JavaScript | pending | |

## Appendix

Spec surface: `network.addDataCollector` (`dataTypes`, `maxEncodedDataSize`,
`collectorType` default `blob`), `network.getData` (`dataType`, `collector?`, `disown?`,
`request`) β†’ `{bytes: BytesValue}`, `network.removeDataCollector`, `network.disownData`.
Timing: `network.RequestData.timings: network.FetchTimingInfo` (thirteen float fields).
Implementation status as of mid-2026: Firefox supports `response` (and added `request`)
with known redirect-timing fixes in progress; Chromium supports data collectors (consumed
by Puppeteer). This unevenness is the reason for the capability gate in the decision.