Most features completed

.github/workflows/ci.yml

@@ -21,7 +21,13 @@ jobs:
       - run: cmp LICENSE-MIT crates/rapidgzip-rust-cli/LICENSE-MIT
       - run: cmp LICENSE-BSD-3-CLAUSE crates/rapidgzip-core/LICENSE-BSD-3-CLAUSE
       - run: cmp LICENSE-BSD-3-CLAUSE crates/rapidgzip-rust-cli/LICENSE-BSD-3-CLAUSE
-      - run: cargo package --workspace --locked
+      # Verify the publishable core tarball builds in isolation.
+      - run: cargo package -p rapidgzip-core --locked
+      # CLI depends on rapidgzip-core via a path dep in this workspace; that always
+      # checks cleanly. Full `cargo package -p rapidgzip-rust-cli` verification
+      # pulls core from crates.io at the same version and only succeeds after that
+      # core release is published (or when packaging both together at release time).
+      - run: cargo check -p rapidgzip-rust-cli --locked
 
   test:
     strategy:
@@ -42,3 +48,18 @@ jobs:
       - uses: dtolnay/rust-toolchain@1.87.0
       - uses: Swatinem/rust-cache@v2
       - run: cargo check --workspace --all-targets --locked
+
+  # Optional ISA-L backend (requires system libisal).
+  isal:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - uses: dtolnay/rust-toolchain@stable
+        with:
+          components: clippy
+      - uses: Swatinem/rust-cache@v2
+      - name: Install libisal
+        run: sudo apt-get update && sudo apt-get install -y libisal-dev
+      - run: cargo test -p rapidgzip-core --features isal --all-targets --locked
+      - run: cargo clippy -p rapidgzip-core --features isal --all-targets --locked -- -D warnings
+      - run: cargo check -p rapidgzip-rust-cli --features isal --locked

.gitignore

@@ -2,6 +2,9 @@
 /target/
 /debug/
 
+# Local package downloads (do not commit)
+*.deb
+
 # Project benchmark and profiling outputs
 /benchmark-results/
 /perf.data
@@ -15,3 +18,7 @@
 
 # RustRover / IntelliJ project settings
 # .idea/
+
+# Local ISA-L install prefix (system or extracted debs)
+.isal-prefix/
+target/isal-prefix/

ARCHITECTURE.md

@@ -3,19 +3,103 @@
 ## Data flow
 
 `rapidgzip-core` accepts an immutable positional `ReadAt` source. A decode
-snapshots its length, parses gzip framing itself, and routes the raw DEFLATE
-payload through one of five bounded paths:
+snapshots its length, detects gzip (`1f 8b`) vs zlib (RFC 1950 CMF/FLG) from
+the prefix (or an explicit `DecoderBuilder::format`), then routes the payload.
+Auto never selects raw DEFLATE (no magic). Non-seekable `Read` sources
+(`Decoder::decode_read`, CLI stdin decompress) use a buffered pull reader:
 
-1. Standard zlib-rs raw inflate is the authoritative fallback and the
-   single-thread path.
-2. A fully stored stream is indexed from its exact block headers and copied by
-   ordered worker tasks.
+- **Sequential streaming** when `decoder_threads == 1`: zlib-rs inflate only; no
+  parallel workers and no full-archive buffer (O(input page) compressed-side
+  memory). Applies to gzip, zlib, and raw DEFLATE.
+- **Spill + positional backend** when `decoder_threads > 1` on non-seekable
+  `decode_read` (any resolved format: gzip, zlib, or raw DEFLATE): the stream is
+  spilled to a private temporary file (secure temp-dir defaults; deleted on
+  drop), then the usual positional path below runs on that file. Peak cost is
+  compressed size on disk plus the decoder working set. After spill, gzip, zlib,
+  and raw DEFLATE use the same parallel gates as file/`ReadAt` input.
+- **Parallel multi-stream zlib** on positional `ReadAt` (including after spill)
+  when `decoder_threads > 1` and the input is ≥2 concatenated independent
+  CMF/FLG…Adler frames: two-pass discard-index of stream boundaries, then ordered
+  worker re-decode with zlib-rs (stream granularity). Skipped when a large single
+  stream will take the marker path (avoids a full discard re-inflate of solitary
+  long streams).
+- **Parallel single-stream zlib** on positional `ReadAt` (including after spill)
+  when `decoder_threads >= 4` and the compressed length is at least about two grid
+  cells (`2 × compressed_chunk_size` plus header/trailer): same estimated
+  marker/window path as ordinary gzip, with Adler-32 trailer checks. Concatenated
+  streams after the first are multi-stream-parallel when ≥1 remain (else
+  sequential).
+- **Parallel single-stream raw DEFLATE** on positional `ReadAt` (including after
+  spill) when `decoder_threads >= 4` and compressed length is at least about two
+  grid cells (`2 × compressed_chunk_size`; no wrapper bytes): same estimated
+  marker/window path; leftover compressed bytes after stream end are an error.
+
+CLI paths that need a known length / `ReadAt` on stdin (`--analyze`,
+`--import-index`, `--ranges`) likewise spill stdin to a tempfile rather than
+holding the full archive only in RAM.
+
+Positional routes after format resolution:
+
+0. **zlib wrapper**: parse CMF/FLG, raw-inflate, verify the big-endian Adler-32
+   trailer (gated by `crc32_enabled`). Supports concatenated zlib streams.
+   Routing when `decoder_threads > 1`: (a) multi-stream stream-granularity
+   parallel if a discard index finds ≥2 frames (unless a large single stream is
+   preferred for the marker path); (b) single long streams use the estimated
+   marker/window path when `decoder_threads >= 4` and size amortizes the grid;
+   (c) multi-stream tails after a parallel first stream use stream-granularity
+   parallel (or sequential for a single remainder); else sequential zlib-rs.
+0b. **raw DEFLATE** (RFC 1951, explicit `Format::RawDeflate` only; Auto never
+   selects it): single long streams use the estimated marker/window path when
+   `decoder_threads >= 4` and size amortizes ~2× the compressed grid; otherwise
+   sequential zlib-rs raw inflate (`windowBits = -15`) from offset 0 to
+   `Z_STREAM_END`. Leftover compressed bytes are an error. No container integrity
+   trailer and no random-access index (`keep_index` rejected at build time);
+   optional whole-stream CRC via `raw_crc32_list`.
+1. Standard zlib-rs raw inflate is the authoritative gzip fallback and the
+   default single-thread gzip path. Paths that go through the crate-private
+   `InflateBackend` trait (generic, monomorphized to `ActiveInflater` — zlib-rs
+   `RawInflater` by default, or ISA-L `IsalInflater` with the optional `isal`
+   feature; see `inflate_backend.rs` / `isal_backend.rs`):
+   - sequential positional multi-member gzip, sequential zlib, sequential raw
+     DEFLATE, and streaming gzip/zlib/raw via `stream_decode` (`create` /
+     `reset` / trait `inflate`);
+   - structure analysis (`Decoder::analyze` / CLI `--analyze`) with
+     `InflateFlush::Block` (block spans + `last_block`; ISA-L backend delegates
+     Block flush to an internal zlib-rs inflater);
+   - parallel BGZF one-shot block inflate via trait `inflate`
+     (`InflateFlush::Finish`; ISA-L multi-steps `isal_inflate` until
+     STREAM_END within that public call when `tmp_out` still drains);
+   - parallel independent-member workers: trait `create`/`reset`/
+     `inflate_capped` so the per-member output budget cannot overshoot spare
+     capacity (no raw `stream.avail_out` at the call site);
+   - multi-stream zlib index discard-inflate and parallel zlib stream workers
+     via trait `create`/`reset`/`inflate`/`inflate_capped`;
+   - mid-stream bit resume first-class on the trait (`prime`,
+     `set_dictionary`, default `install_bit_resume` /
+     `prepare_at_bit_offset`): estimated-path setup uses trait
+     `prime`/`set_dictionary`, and source-backed seeks /
+     `inflate_from_block` build streams via
+     `prepare_inflater_at_bit_offset` (generic over `I: InflateBackend`);
+   - estimated-path residual continue (`inflate_from_block`) and
+     `inflate_tail` via trait `inflate_capped` (`NoFlush` / `Block`;
+     block-end bit state from `InflateStep`);
+   - seek sessions and indexed segment decode via trait
+     `inflate_into_slice` (fixed caller `out` slices; member restarts use
+     trait `create`/`set_dictionary`).
+
+   All sequential/block inflate call sites go through `InflateBackend`
+   (monomorphized to `ActiveInflater`). Raw `z::inflate` lives in the zlib-rs
+   implementor in `inflate_backend.rs`; ISA-L `isal_inflate` lives in
+   `isal_backend.rs`. Build the `isal` feature only when a shared `libisal` is
+   available (`ISAL_INSTALL_PREFIX` / system packages).
+2. A fully stored gzip stream is indexed from its exact block headers and
+   copied by ordered worker tasks.
 3. A consistently formed BGZF stream is indexed from `BC/BSIZE` and its
    independently verified members are decoded by ordered worker tasks.
-4. Ordinary streams with densely spaced members use candidate-header
+4. Ordinary gzip streams with densely spaced members use candidate-header
    discovery and independently verified member workers.
-5. Other streams use a file-wide estimated grid and rapidgzip's marker/window
-   path, with zlib-rs fallback from the last authoritative boundary.
+5. Other gzip streams use a file-wide estimated grid and rapidgzip's
+   marker/window path, with zlib-rs fallback from the last authoritative boundary.
 
 All paths return ordered owned chunks to one coordinator. The coordinator alone
 updates member accounting and calls the user's `Write`, so a writer need not be
@@ -106,7 +190,9 @@ FASTQ members from losing parallelism or inflating result-buffer residency.
 
 CRC32 and modulo-2^32 ISIZE are tracked and checked per member. History resets
 to empty after every footer. Empty members, concatenated gzip, and BGZF EOF
-members therefore use the same semantics.
+members therefore use the same semantics. Sequential multi-member gzip and
+zlib paths (positional and streaming) reuse one raw-inflate stream across
+members via `inflateReset`, which clears window history at each boundary.
 
 The BGZF route is selected only when every declared `BSIZE` leads exactly to
 another `BC` header or EOF. Mixed BGZF/plain streams and gzip members with an
@@ -141,10 +227,18 @@ Each candidate uses the median of three intervals. Work carries a controller
 generation, and completions begun under an earlier limit do not inflate the new
 candidate's byte count. The search probes downward first, preferring a lower
 setting within a 3% noise margin, or climbs in quarter-bootstrap steps while
-throughput materially improves. Its empirical search extent is at most twice
+throughput improves by more than 5% (stricter than the down-probe tolerance so
+noise does not keep adding ranks). Its empirical search extent is at most 1.5×
 the bootstrap and never exceeds the configured budget. This bounds calibration
 and speculative-memory exposure without a compiled-in worker cap.
 
+Ordinary single-member gzip **and** large single-stream zlib enter the
+estimated/marker pipeline only when the requested budget is at least four
+workers; at two or three threads sequential zlib-rs is faster end-to-end
+(measured). Zlib also requires enough compressed bytes for at least two grid
+cells (see zlib route above). BGZF, stored-block, and independent multi-member
+gzip paths still parallelize for any budget greater than one.
+
 The active limit also controls the decode/resolve scheduling horizon because
 each speculative result commonly owns several MiB of `u16` symbols. Workers
 have stable ranks and are created lazily as upward probes require them. Ranks
@@ -160,10 +254,74 @@ At each ordinary gzip member transition, the coordinator resets
 history/accounting and decodes an exact bridge from the new header to the first
 later file-wide grid point; already-running tasks beyond that point remain
 useful. Results and resolved buffers are reordered by ordinal before being
-committed. No speculative worker calls the user's output object.
+committed. No speculative worker calls the user's output object. After a
+successful resolve+emit, empty `Vec<u8>` capacity (marked resolved bytes,
+clean, backend tail) is returned to a soft-capped free-list
+(`2 × worker_count`) so workers can reuse it for the next task's clean/tail
+scratch. After resolve on an estimated worker, emptied `Vec<Symbol>` capacity
+is recycled into that worker's `marked_scratch` (resolve clears symbols and
+returns the allocation; never while the buffer is still on the resolve queue).
 
 Input is paged with positional reads. Speculative output is capped per task;
 oversized regions continue through zlib-rs instead. `DecoderReader` adds at most
 the configured in-flight chunk count plus its currently partially consumed
-chunk. Dropping it closes the consumer edge, sets cancellation, and joins the
-coordinator.
+chunk, and shares a soft-capped reader-local free-list (`2 × in_flight_chunks`,
+`buffer_pool::ByteBufferFreeList`) with `ChannelOutput`: fully-consumed channel
+chunks are recycled so `emit_reusable` can return capacity for the next fill.
+That pool is separate from the estimated-path free-list. Dropping the reader
+closes the consumer edge, sets cancellation, and joins the coordinator.
+
+## Structure analysis
+
+`Decoder::analyze` (CLI `--analyze`) walks the archive sequentially via
+`InflateBackend` with `InflateFlush::Block` (monomorphized to `RawInflater`).
+Format follows `DecoderBuilder::format` (default auto-detect): gzip/BGZF, zlib
+(RFC 1950), or explicit raw DEFLATE (RFC 1951). It reports per-member (or per
+stream) compressed ranges, uncompressed sizes, footer check status, and
+per-block type (stored/fixed/dynamic), final bit, and bit spans. For zlib it
+also records CMF/FLG and Adler-32 status (`crc32_enabled` gates Adler the same
+way as gzip CRC). Concatenated zlib streams appear as separate members. No
+payload is emitted and no index is required.
+
+Raw DEFLATE (`Format::RawDeflate`, never auto-selected) walks a single stream
+from bit 0 to `Z_STREAM_END`, reports `ArchiveKind::RawDeflate` with
+`crc32_ok: None` (no integrity trailer), and treats trailing bytes after EOS as
+an error (same policy as decode).
+
+## Random access and indexes
+
+When `DecoderBuilder::keep_index` is enabled, the coordinator records
+checkpoints (compressed bit offset, uncompressed offset, optional line
+offset) and predecessor 32 KiB windows at resolved boundaries. BGZF and
+independent member starts use empty windows. By default
+(`compress_index_windows`, on), non-empty windows may be held
+zlib-compressed in memory when smaller than raw (decompress on demand for
+seek/export; `IndexedReader` caches expanded zlib windows in a small LRU
+aligned with seek-cache chunk budget). The in-memory `GzipIndex`
+exports/imports **indexed_gzip** (`GZIDX`), **gztool** (`gzipindx` /
+`gzipindX`), and htslib **BGZI** (`.gzi` block index: little-endian pair
+list of compressed/uncompressed block starts after the first). BGZI export
+emits only empty-window member boundaries (no synthetic EOF pair, no
+mid-stream windows). Import leaves uncompressed size unknown (`u64::MAX`);
+full decode schedules an open-ended final segment for the last block.
+`read_gzip_index` auto-detects the format (GZIDX magic, then gztool magic,
+then exact-length BGZI).
+
+`Decoder::decode_with_index` splits the archive on consecutive checkpoints and
+inflates each span with zlib-rs (no marker speculation). Segments may start
+mid-member, so this path does **not** verify member CRC32/ISIZE (same policy
+as seek). Self-built indexes append an EOF checkpoint; imported indexes that
+omit one still get a final tail segment when the declared uncompressed size is
+known. Empty-window checkpoints that land on gzip magic (BGZI header starts)
+skip the member header so inflate begins at the DEFLATE payload; marker-path
+fallbacks never perform that skip.
+
+`IndexedReader` (`Read` + `Seek`) restarts inflate at the nearest preceding
+checkpoint, discards skip bytes, and serves sequential traffic from an LRU of
+decoded windows with optional single-threaded readahead into the next window
+plus best-effort parallel background prefetch of further windows
+(`DecoderBuilder::seek_prefetch_windows`, default 2; workers inflate from
+independent checkpoint resumes and never share the consumer session). Far seeks
+invalidate in-flight prefetch inserts via a generation counter. `seek_to_line`
+requires an index with line offsets (`gather_line_offsets` or a
+gztool-with-lines import).