Text audit#614
Merged
Merged
Conversation
Add text/audit.md mirroring dev/audit.md: a per-utility checkbox punch list scoring each text-crate utility against its POSIX.1-2024 spec slice, following the audits.md playbook. One read-only subagent per utility cross-referenced spec, implementation, and tests; Critical/Major findings were verified against cited line ranges. Headline findings: join is an early-stage stub needing near-total rework; sed (l command unimplemented, ERE flag set after compile, a/r not deferred) and sort (no strcoll collation, -m concatenates rather than merges, broken -u) have multiple golden-path defects; csplit, cut, unexpand, tail, fold, uniq, expand carry data-affecting bugs. asa is essentially conformant. Cross-cutting patterns: (1) the `-` operand is widely not routed to stdin (dashed_stdin: false); (2) locale is initialized but unused (no strcoll, ASCII-only character classes, no wcwidth); (3) runtime diagnostics are hardcoded English; (4) Issue 8 additions tail -r, tsort -w, sed s///i are missing; (5) several test suites never exercise the core transformation, hiding real bugs. Audit only; no utility code modified. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add wcwidth_char(char)->i32 (libc wcwidth) for display-column width and nine LC_CTYPE-aware character-class predicates (isblank, isspace, isalpha, isalnum, isdigit, ispunct, iscntrl, isgraph, isxdigit) via a ctype_predicate! macro mirroring isprint's byte-vs-wide dispatch. Wide-character libc functions are declared directly (extern "C") since the libc crate does not surface them on all targets, matching the existing iswprint/mbrtowc precedent; cfg-correct for macOS via the existing WintT alias. Foundation for the text/ audit fixes: expand/fold/unexpand/pr column-width (wcwidth), and tr/sort/uniq/wc character classification (ctype predicates). Unit tests cover the C-locale ASCII classifications and a UTF-8 wide-character width (世 = 2 columns) gated on locale availability. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
… utilities
POSIX (XBD §12.2 Guideline 13) accepts '-' as a stdin operand at any position
in the file list, not only when it is the sole operand. Add
plib::io::input_stream_dashed (empty path or "-" -> stdin, else open the file)
and use it so a '-' interleaved with real files reads stdin at that position:
- expand, fold, head, wc, tsort: '-' (and the no-operand case) now read stdin
instead of failing to open a file literally named "-".
- cut, sort, unexpand: '-' is honored at any position, not just as the sole
operand.
- uniq: a '-' output_file operand now writes to stdout, not a file named "-".
The dashed helper returns the unlocked io::Stdin handle (which locks per read)
rather than a persistent StdinLock, so a utility that opens several stdin
sources at once (e.g. `cut - -`, `sort - -`) does not deadlock acquiring the
stdin lock twice; the first source drains stdin and later ones see EOF.
Closes audit items: expand #7, fold #5, head #1, tsort #3, wc #2, cut #2,
sort #21, unexpand #8, uniq #3. Adds a regression test per utility plus
plib::io::input_stream_dashed unit tests; full text suite (728) still passes.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
expand: rewrite the tab expander over locale-segmented characters with a
stateless 0-based column. This fixes four bugs at once:
- `-t 0` (and `0` in a `-t` list) now error instead of panicking on
modulo-by-zero;
- an explicit `-t` stop list no longer overshoots each stop by one column
(a leading `-t 4` tab is 3 spaces, landing at position 4);
- the stop index resets at every newline, so lines after the first expand
correctly;
- tab/column width follows wcwidth(3) under LC_CTYPE, so a 1-column
multibyte character advances the column by one, not by its byte length.
fold: rewrite the folder over locale-segmented characters.
- `-s` now breaks after the genuine last <blank> within the width
(find_last_blank returned a reversed offset used as a forward index,
corrupting every space-mode fold);
- <blank> is restricted to space and tab (POSIX), not Rust is_whitespace;
- column width follows wcwidth(3); wide characters are never split
mid-character.
Both read input via plib::locale::mb_char_slices (LC_CTYPE segmentation) and
plib::locale::wcwidth_char. The non-POSIX `--tablist`/`--bytes`/`--spaces`
long options are kept and documented.
Closes audit items: expand #1-#5, fold #1-#4. Adds regression tests for
zero-width rejection, multi-stop lists, per-line reset, off-by-one, -s blank
breaks, hard breaks, and (locale-gated) multibyte width.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…/POSIX A leading tab under `-t 4,8` advances to column 4 (four spaces), not three: the list values are 0-based column positions on the same scale as the uniform multiples, so `expand -t 4` and `expand -t 4,8` share that first stop. The prior next_stop List branch used `s - 1` (a 1-based reading) and the three list tests encoded the off-by-one. Verified byte-for-byte against GNU coreutils 9.4. Also marks the expand #1-#6 and fold #1-#4,#6 audit items closed. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Replace the per-run, leading-only converter with a single-pass converter
(unexpand_line) that tracks the true 0-based column across the whole line:
- default and single-integer `-t N` tab stops now repeat every N columns,
so 16 leading spaces under the default 8-column stops become two tabs
(the stop list previously did not repeat);
- `-t` implies `-a` (POSIX): conversion is not limited to leading blanks;
- `-a` converts runs of two or more spaces from their actual column
position (the old code measured each run from column 0 and used only the
first tab stop);
- existing `<tab>` characters advance the column and do not end the
leading-blank region; `<backspace>` is copied and decrements the column;
- the tab list accepts comma- or blank-separated values and rejects zero;
- <blank> is space and tab only (not Rust is_whitespace), and column width
follows wcwidth(3) under LC_CTYPE via plib::locale.
Tab-stop list values are 0-based column positions on the same scale as the
uniform multiples, matching expand and GNU coreutils 9.4 (verified
byte-for-byte). Input lines are read as raw bytes so exact line endings and
non-UTF-8 bytes are preserved; a missing final newline is no longer
fabricated. The four existing tests that encoded the old behavior were
corrected, and tests added for zero rejection, blank-separated lists,
interior-preservation, and multibyte width.
Closes audit items: unexpand #1-#7, #9, #10.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…xed skips
Add plib::locale::MbDecoder, a streaming mbrtowc-based multibyte decoder that
carries a partial sequence across read boundaries.
wc:
- every file operand (including "-") prints its name; only the no-operand
standard input omits it (previously a single named file showed no name);
- "-m" counts characters under LC_CTYPE via MbDecoder, and "-w" splits words
using libc iswspace on decoded characters (the ASCII-only whitespace table
missed locale whitespace such as U+3000);
- the numeric field width adapts to the largest value printed across files
and the total, instead of a fixed width of 8;
- a file that fails to open contributes no row and is excluded from the total.
uniq:
- "-s" skips by character index, not byte offset, so multibyte input no
longer panics on a non-char-boundary index;
- field skipping consumes each field as [[:blank:]]*[^[:blank:]]* (leading
blanks belong to the field) using plib::locale::isblank;
- when -f/-s consume the whole line the comparison key is the empty string,
so such lines compare equal;
- the comparison key is computed once per line.
The -c|-d|-u mutual-exclusion errors are kept (POSIX leaves the combination
undefined; a diagnostic is conforming).
Closes audit items: wc #1,#3,#4,#5,#6; uniq #1,#2,#5,#6 (and #4,#7 accepted).
Verified byte-for-byte against GNU coreutils 9.4; adds regression tests
including locale-gated multibyte cases and plib MbDecoder unit tests.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…cycles comm: compare lines in the LC_COLLATE collating sequence (plib::locale::strcoll) with a byte-comparison tiebreak for collate-equal-but-distinct lines (POSIX Issue 8). head: -n 0 / -c 0 now produce empty output and exit 0 (the count is an application constraint, not a utility error), while still printing multi-file headers. paste: serial mode (-s) opens each operand at processing time and continues past a file that fails to open (POSIX Section 1.4), instead of aborting the whole run; parallel mode keeps its eager open. tsort: implement -w (exit status = number of cycles, capped at 255); detect and break cycles with DFS so the total order still covers every node, report each cycle as its own group on stderr, and make any cycle a non-zero exit (the three cycle tests now assert exit 1). asa/comm/paste/head/tsort: route diagnostics through plib::diag (init_locale + error + exit_status) for a uniform "<util>:" stderr prefix and correct exit status; the asa "-" triple-spacing extension is kept and documented. Closes audit items: asa #1(accepted),#2; comm #1,#2(accepted),#3; head #2,#3(accepted),#4; paste #1,#2,#6 (#3-#5 accepted); tsort #1,#2,#4(accepted),#5,#6. Verified against GNU coreutils 9.4; adds tsort -w tests. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
… on error - -f without -d now uses <tab> as the default field delimiter (previously the whole line was echoed verbatim); - input lines are read as raw bytes and -b writes the selected bytes verbatim, so it handles bytes that are not valid UTF-8 (the result was previously routed through a String and dropped on invalid UTF-8); - -n snaps byte ranges to LC_CTYPE character boundaries (via plib::locale::mb_char_slices), snapping low to a character start and high to a character end and dropping a collapsed range — a no-op in the C locale, the POSIX snap rule under a multibyte locale (GNU coreutils ignores -n, so it is not a reference here); - lists accept blank- or tab-separated values, not only commas, and the single-number detection uses the split-element count; - fields are split on the raw delimiter character (dropping the escape_debug transformation that broke a backslash delimiter); - each operand is opened at processing time, so a file that fails to open is diagnosed via plib::diag and processing continues, affecting only the exit status. Closes audit items: cut #1,#3,#4,#5,#6,#7. Adds regression tests for the default tab, blank-separated lists, raw byte output, and continue-on-error; existing field/byte/char tests still pass. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ejection tr is fully POSIX-conformant in the C/POSIX locale (the conformance baseline): there -c and -C are equivalent, [:class:] expands to the documented ASCII sets, each character is its own equivalence class, and code-point order is the collation order. The remaining audit findings (#1 -c/-C, #2 locale classes, #4 [=equiv=] expansion, #5 collation-order ranges) concern non-POSIX multibyte locales; a correct fix requires replacing the finite-set class expansion with predicate-based matching and special-casing [:upper:]/[:lower:] case conversion (currently positional pairing of two 26-element vectors), an architectural change deferred as a documented POSIX-locale limitation (the sanctioned fallback for tr's locale items). Resolved the locale-independent items: #3 (other class names in string2 are left undefined by POSIX, so accepting them conforms) and #6 (added a regression test that [c*] in string1 is rejected). #7 (backslash before a non-octal char) is accepted per the spec's "unspecified" latitude. Closes audit items: tr #3,#6,#7 (resolved); #1,#2,#4,#5 documented as POSIX-locale limitations. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
grep:
- keep every specified pattern (removed the sort_by_key + dedup that dropped
duplicates, which could change the exit status, e.g. when an empty match-all
pattern is supplied twice);
- -F -i case-insensitive matching folds via plib::locale::to_lower (libc
tolower/towlower), honoring LC_CTYPE instead of Rust Unicode folding;
- diagnostics route through plib::diag (uniform "grep:" prefix on stderr),
matching the GNU/POSIX convention; existing tests updated accordingly.
nl:
- the pBRE numbering style now uses the libc-backed plib::regex BRE engine, so
it is a Basic RE (`+`/`?`/`{}` literal, `\(…\)`/`\{n\}` operators) and bracket
expressions honor LC_COLLATE/LC_CTYPE, instead of the ERE/Perl `regex` crate;
- the -l blank-line run counter resets when crossing a logical-page section
delimiter.
Closes audit items: grep #1,#2,#3 (#4,#5,#6 accepted); nl #1,#2,#3,#4
(#5,#6 accepted). Adds regression tests for duplicate-pattern retention and
nl BRE semantics (literal `+`, `\{n\}` interval).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…t splits
The split loop dropped each file's trailing newline (corrupting output and the
reported byte count), reset the line counter after every split (breaking
absolute line numbers across multiple operands), and never consumed a bare
line-number operand (so it split repeatedly to EOF). Reworked it so:
- `in_line_no` is a monotonic absolute counter; `LineNum { target, step }`
carries the repeat step, so `csplit f 5 10` splits at absolute lines 5 and 10
and a bare `csplit f 5` splits exactly once.
- `{N}` fires N+1 times (base + N repeats), matching GNU; `Repeat(usize::MAX)`
(`{*}`) no longer risks an underflow.
- the trailing-newline `pop()` calls were removed, so files keep their bytes and
the printed count equals `wc -c`; the negative-offset path was rewritten to
move whole lines preserving newlines.
- escaped delimiters (`\/`, `\%`) are translated to a literal before the BRE is
compiled, so `/proc\/sys/` matches `proc/sys`.
Output verified byte-for-byte against GNU coreutils 9.4 (line numbers, repeats,
positive/negative regex offsets, suppress). Test expectations updated to the
now-correct byte counts and 5-file split counts.
Closes audit items: csplit #1,#2,#3,#5,#6 (fixed); #7 accepted. Remaining:
#4 (SIGINT cleanup handler) and #8 (error on operand past EOF) — deferred.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…y, stderr - -b: verified the blank-normalization matches GNU (interior/leading runs collapse to one space, trailing blanks stripped, leading presence significant); added regression tests. - existence-check diagnostics now go to stderr (eprintln), not stdout. - -r: track visited (dev,ino) so symlink cycles terminate instead of hanging; also propagate subdirectory exit status (was discarded). - context format: single-line ranges print `*** N ****` (not `N,N`); empty ranges print the preceding line number; verified vs diff -c. - unified header timestamps include fractional seconds and the timezone offset (`%Y-%m-%d %H:%M:%S%.9f %z`). - -C 0 / -U 0 accepted (removed the clap range floor + n==0 clamp); zero-context output matches GNU. - "no newline at end of file" marker is emitted inline after the file's actual last newline-less line (default, context, and unified formats), keyed on the real last line rather than hunk position; for -e/-f it goes to stderr so the ed script is not corrupted. - -f multi-line ranges are space-separated (`c2 4`); default/ed keep comma. - directory compare skips FIFO/block/char special files instead of blocking. All output verified byte-for-byte against GNU diffutils. 28 diff tests pass (9 new regression tests), clippy clean. Closes audit items: diff #1-#11. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- header date uses %e (space-padded day) instead of %d, per spec. - -p pause reads the response from /dev/tty (not stdin) and triggers on <newline> (\n) per Austin Group Defect 1433, instead of \r off stdin. - -f (XSI form-feed pause) pauses only before the first page via a dedicated flag, instead of reusing -p's every-page pause. - -e (input-tab expansion) is implicitly assumed for multi-column output per spec; -i is intentionally NOT implicitly assumed (GNU pr emits spaces, not tabs, for multi-column output — assuming -i would break byte-for-byte parity). - -s no longer requires multi-column mode (pr -s file works single-column). - form-feed mid-page returns the real non-padding line count so multi-column row math is correct. - a SIGINT handler (installed only when stdout is a tty) flushes output and re-raises with default disposition. pr tests 21/21 pass, clippy clean. Closes audit items: pr #1-#8. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
… codes
Reworked the comparison engine (decorate-sort: each line precomputes per-key
comparison values) to address the algorithmic defects:
- text comparison uses plib::locale::strcoll (LC_COLLATE), not Rust byte order;
the whole-line last-resort tiebreak is strcoll-then-byte ("as if no options
but -r"), verified vs GNU in C and en_US.utf8 locales.
- -m performs a true stable k-way merge instead of concatenating.
- -u dedupes by comparing adjacent sorted lines on the key comparator (works
for zero keys and 3+ keys); removed the broken side-channel.
- -n rejects '*', strips leading blanks, and compares with arbitrary precision
(sign, length-then-lexicographic integer, padded fraction) so values beyond
2^53 are exact; locale radix/grouping read via localeconv.
- all -k keys (>=9) are processed, each with its own modifiers; a whole-line
tiebreak applies when all keys are equal.
- global -b and -t (without -k) are accepted.
- -C is silent; -c/-C scan sequentially and report the first out-of-order line
with GNU's "FILE:LINE: disorder: TEXT" message.
- -i/-d use LC_CTYPE (isprint / isblank+isalnum) instead of ASCII/Unicode.
- default field separator treats SPACE and TAB as <blank>.
- start/end key modifiers parsed independently; usize::MAX-1 sentinel replaced
with Option<usize>.
- -o/-m read all inputs before opening the output (no self-truncation).
- argument/operand errors exit 2; exit 1 reserved for -c/-C disorder.
- removed the false -d/-i, -d/-n, -n/-i mutual-exclusions (POSIX-conformant;
GNU rejects two of these — a deliberate divergence).
All behavior verified byte-for-byte against GNU coreutils 9.4 under C and
en_US.utf8. 116 sort tests pass (10 new regression tests + merge fixtures),
clippy clean.
Closes audit items: sort #1-#20, #22 (#21 done in Phase 2).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- implement the POSIX l command (three-digit octal escapes, C escapes, \\, $ terminator, fold at width with trailing \, optional width arg); keep the non-POSIX I extension sharing the octal renderer. - set the ERE flag before Script::parse so -E patterns compile as ERE. - a and r write through a deferred-output queue flushed before the next input read, so later commands in the cycle no longer see the appended/read text. - D no longer clears the hold space. - add the s///i case-insensitive flag (REG_ICASE), mandatory since POSIX.1-2024. - = prints under -n. - pre-create/truncate all wfiles at startup; w/s///w relative paths resolve against the CWD instead of CARGO_TARGET_TMPDIR. - resolve y/// escapes at parse time (no pattern-space post-processing); convert s/// replacement escapes (\n, \t, \\, \&, backrefs) correctly. - support the multi-line a\/i\/c\ POSIX text form; positional scripts now preserve embedded newlines and treat a bare newline as a command separator. - accept duplicate :label definitions (matching GNU). All behavior verified against GNU sed 4.9. 81 sed tests pass (11 new regression tests), clippy clean. Closes audit items: sed #1-#16 (#11,#12,#14 accepted as GNU-correct). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…hort-read - implement -r (reverse): whole file by default, limited by -n/-c; per POSIX.1-2024 Issue 8 / BSD. - replace the inotify (notify-debouncer) follow with a sleep-poll loop, so -f works on a FIFO operand (blocking reads) and no longer exits when the followed file is removed (log rotation); removed the now-unused notify-debouncer-full dependency. - route diagnostics through plib::diag (uniform "tail:" prefix) with gettext-wrapped messages. - print_n_bytes accumulates until a true EOF (zero-byte read) instead of stopping on any short read. - verified -- still ends options with allow_hyphen_values; added regression tests for -r, -- handling, and -n +0 (which prints all, matching GNU). -n +0 is accepted as GNU-correct (GNU treats +0 like +1). FIFO/removal follow behavior verified manually. 32 tail tests pass, clippy clean. Closes audit items: tail #1-#7 (#3 accepted as GNU-correct). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ersed detect - track the "\ No newline at end of file" marker and suppress the final newline when the patched file ends without one (was silently appending a newline). - a +++ /dev/null deletion patch removes the target (backing it up first under -b) instead of writing an empty file. - implement the interactive "File to patch:" prompt on /dev/tty when the target cannot otherwise be resolved. - forward/reverse auto-detection: prompt "Reversed (or previously applied) patch detected! Assume -R?" on /dev/tty; -R reverses directly, new -f assumes yes. - -o concatenates successive versions (truncate first, append after); -b backs up each file only the first time it is patched (preserving the true original). - -p collapses runs of '/' before counting components; common leading-blank prefix stripped before parsing. - -D emits the standard #ifndef/#else/#endif form; reject-file hunk headers are shifted by the cumulative applied-hunk offset. - fixed a multi-file unified-patch splitting bug (hunk lines bounded by the header counts) that prevented per-file backup/concatenation. All data-mutating behavior verified byte-for-byte against GNU patch. 34 patch tests pass (5 new + ifdef updated), full text suite 796 pass, clippy clean. Closes audit items: patch #1-#10. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Replaced the O(N×M) nested-rescan stub with a proper merge-join porting GNU coreutils join semantics: - default output is the join field once, then file1's other fields, then file2's other fields — correct for any -1/-2 (was leaving the join field in place and dropping file2's first field). - single-pass merge over equal-key groups with full cartesian product on duplicate keys; key comparison uses plib::locale::strcoll (LC_COLLATE). - default separator splits on maximal non-blank runs (space+tab collapse, leading blanks ignored); -t char is the single input AND output separator. - -a and -v are repeatable (-a 1 -a 2, -v 1 -v 2) with correct unpairable bookkeeping for duplicate keys; -o supports 0 (join field, no longer panics), comma- and blank-separated lists, and multiple -o args; -e fills missing -o fields including on unpairable lines; output honors -t throughout. - '-' works as either file via input_stream_dashed (both-stdin is an error). - assert!/panic! replaced with plib::diag diagnostics and non-zero exit. Verified byte-for-byte against GNU join 9.4 across 34 differential cases plus ~23,000 randomized/disordered/UTF-8 fuzz cases (0 mismatches). Corrected the third_field_join_test expectation (it encoded the old buggy reconstruction); added 14 GNU-verified regression tests. 23 join tests pass, clippy clean. Closes audit items: join #1-#12. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The tail -f rewrite (sleep-poll) removed the notify-debouncer-full dependency; this prunes notify and its transitive deps (inotify, fsevent-sys, kqueue, mio, parking_lot, windows-sys, ...) from the lockfile. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The text/ POSIX audit remediation (18 phases on this branch) is complete. Move the 20 utilities whose findings are all resolved from Stage 3 to Stage 6 (Audited): asa, comm, cut, diff, expand, fold, grep, head, join, nl, paste, patch, pr, sed, sort, tail, tsort, unexpand, uniq, wc. csplit and tr stay at Stage 3: - csplit: #4 (SIGINT created-file cleanup) and #8 (error on operand past EOF) remain open; its file output is already byte-for-byte GNU-correct. - tr: fully POSIX-conformant in the C/POSIX locale; the multibyte items (-c/-C, LC_CTYPE classes, [=equiv=], LC_COLLATE ranges) are documented POSIX-locale limitations needing a predicate-based class-matching rewrite. Adds the text/ entry to audits.md and per-utility STATUS banners to text/audit.md. Whole-workspace cargo test --release: 7663 passed, 0 failed. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
jgarzik
added
bug
There was a problem hiding this comment.
Pull request overview
This PR is a broad POSIX/text-utility conformance audit sweep across the text/ crate, aligning behavior and diagnostics with POSIX (and frequently GNU/BSD reference behavior), and adding/adjusting integration tests to lock in the corrected semantics.
Changes:
- Improve operand handling and locale behavior across multiple utilities (notably treating
-as stdin at any position and making multibyte/ctype behaviorLC_CTYPE-aware in several places). - Tighten POSIX/GNU-compatible output formats, exit statuses, and diagnostics for tools like
diff,patch,pr,tsort,sort, etc. - Expand integration test coverage substantially with regression tests for the audited findings.
Reviewed changes
Copilot reviewed 63 out of 65 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
| text/wc.rs | Locale-aware -m/-w decoding + POSIX - operand handling and output formatting refactor. |
| text/uniq.rs | Correct - output operand semantics and make skip logic character-safe / POSIX field rules. |
| text/unexpand.rs | Rework tab-stop parsing, stdin handling, and LC_CTYPE-aware column logic; preserve exact line endings. |
| text/tsort.rs | Add -w, cycle detection/reporting changes, and migrate locale/diagnostics handling. |
| text/tests/wc/mod.rs | New wc regression tests (dash operand, filename printing, locale-aware -m/-w). |
| text/tests/uniq/mod.rs | New uniq regression tests (dash output, multibyte -s, POSIX field rules). |
| text/tests/unexpand/mod.rs | Expanded unexpand tests for newline handling, -t implies -a, tablist formats, locale width. |
| text/tests/tsort/mod.rs | Update cycle exit status expectations; add stdin-dash and -w tests. |
| text/tests/tr/mod.rs | Add regression test for rejecting repeat construct in string1. |
| text/tests/tail/mod.rs | Add tests for BSD -r behavior and -- option termination edge cases. |
| text/tests/sort/mod.rs | Update/add tests to match audited GNU/POSIX behaviors (tie-breaking, diagnostics, exit codes, merge). |
| text/tests/pr/mod.rs | Update date formatting expectation and add regressions for -s single-column + form-feed accounting. |
| text/tests/patch/mod.rs | Strengthen patch assertions and add multiple new behavioral regressions (no newline, delete, concat, backup, -p). |
| text/tests/nl/mod.rs | Add tests ensuring pBRE uses BRE semantics (not ERE). |
| text/tests/join/mod.rs | Major join test harness refactor + many new GNU-verified expectations (stdin operands, -o, sorting diagnostics). |
| text/tests/head/mod.rs | Add regression test for - reading stdin. |
| text/tests/grep/mod.rs | Adjust expected diagnostics to include program prefix and add duplicate-pattern regression. |
| text/tests/fold/mod.rs | Add stdin-dash regression and additional folding behavior tests (spaces, tabs, locale width). |
| text/tests/expand/mod.rs | Add stdin-dash regression and new tablist/locale-width tests. |
| text/tests/diff-tests.rs | Add a large suite of GNU-verified POSIX diff regressions and helper utilities. |
| text/tests/cut/mod.rs | Add regressions for dashed stdin ordering, default delimiter, blank-separated list, raw bytes, and error continuation. |
| text/tests/csplit/mod.rs | Update expected split byte counts and cleanup for audited behavior changes. |
| text/tests/assets/sort_merge_a.txt | New sort merge fixture file. |
| text/tests/assets/sort_merge_b.txt | New sort merge fixture file. |
| text/tests/assets/sort_merge_c.txt | New sort merge fixture file. |
| text/tests/assets/sort_merge_d.txt | New sort merge fixture file. |
| text/pr.rs | Fix pause behavior to read from /dev/tty, adjust formatting behavior, and add SIGINT handling + XSI pause semantics. |
| text/pr_util/page_iterator.rs | Correct accounting for early page termination by form-feed while keeping layout padding. |
| text/pr_util/args.rs | Allow -s in single-column, model XSI pause-first-page, and implicitly enable -e for multi-column output. |
| text/patch.rs | Add -f, deletion handling, correct -o concatenation and backup-once semantics, and newline-at-EOF tracking. |
| text/patch_util/unified.rs | Fix unified parsing edge cases (no-newline markers, multi-file boundaries, hunk line consumption). |
| text/patch_util/types.rs | Track no-newline markers per hunk and carry them through reverse/apply results; add force to config. |
| text/patch_util/parser.rs | Implement common leading-blank prefix stripping for quoted/indented patches. |
| text/patch_util/normal.rs | Correctly attribute no-newline markers to old/new sides. |
| text/patch_util/file_ops.rs | Add /dev/tty prompts, POSIX -p slash-collapsing, backup-once, deletion, and -o concatenation support. |
| text/patch_util/context.rs | Track no-newline markers for context diffs and propagate to unified-like hunks. |
| text/patch_util/applier.rs | Add reversed-patch detection/prompting, reject header adjustment, ifdef change-block formatting, and EOF-newline preservation. |
| text/paste.rs | Refactor operand opening (stdin sharing), align error-consequence semantics for serial vs parallel, and use plib::diag. |
| text/nl.rs | Switch to libc-backed BRE regex engine and fix display formatting + section delimiter behavior. |
| text/head.rs | Use dashed stdin opener, adjust -n 0 behavior, and migrate diagnostics to plib::diag. |
| text/grep.rs | Use plib::diag, preserve duplicate patterns, and implement locale-aware case folding for -F -i. |
| text/fold.rs | LC_CTYPE-aware width logic, dashed stdin opener, output buffering, and new folding algorithm. |
| text/expand.rs | LC_CTYPE-aware column advancement, tablist validation, and dashed stdin opener. |
| text/diff.rs | Permit -C 0 and adjust context/unified handling accordingly. |
| text/diff_util/hunks.rs | Correct range formatting, and fix “no newline at end of file” marker placement (incl. ed-script stderr handling). |
| text/diff_util/functions.rs | Update unified timestamp format and ensure missing-file diagnostics go to stderr. |
| text/diff_util/file_diff.rs | Fix context/unified range formatting, avoid headers on identical files, and handle ed-script trouble cases. |
| text/diff_util/file_data.rs | Remove filename-only helper in favor of full path usage. |
| text/diff_util/dir_diff.rs | Add recursion cycle protection and skip special files that can block on open. |
| text/cut.rs | Proper - stdin operand handling in-order, raw-byte processing for -b, POSIX list parsing, and diagnostics updates. |
| text/csplit.rs | Fix absolute line-number semantics, repeat behavior, escaped delimiters, and negative-offset handling. |
| text/comm.rs | Use LC_COLLATE ordering via strcoll and migrate diagnostics. |
| text/Cargo.toml | Remove unused dependency entry. |
| text/asa.rs | Migrate locale/diagnostics initialization to plib::diag. |
| README.md | Update utility checklist/stage markers to reflect audited status. |
| plib/src/io.rs | Add input_stream_dashed helper for consistent -/empty-path stdin semantics + tests. |
| audits.md | Add the text/audit.md summary entry and status/promotion notes. |
…, streaming - pr: handle_sigint is now `extern "C"` (a Rust-ABI function registered as a C signal handler is undefined behavior when SIGINT is delivered); register it via a function-pointer cast to sighandler_t. - patch: prompt_yes_no matches a non-empty answer against the locale's YESEXPR (plib::locale::is_affirmative) instead of a hardcoded 'y'/'Y'. - expand, fold: process input one line at a time via a read_until loop instead of read_to_end, restoring bounded-memory streaming for large inputs/pipes (a <newline> never splits a multibyte character, and the per-line/column state resets at each newline, so the output bytes are identical). The unexpand review note was checked and not actioned: the code does not turn a single leading space into a tab (it matches its doc comment); the only related divergence is the conservative direction on the rare `-t 1` case. posixutils-text suite: 875 passed, 0 failed; clippy clean, fmt clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.