Trim adjacent duplicates, count occurrences and tame noisy text files with a single small tool. 21.05.2026 | reading time: 2 min When log files, CSV exports or simple text lists contain repeated lines, the small program "uniq" does one job very well: it collapses adjacent duplicate lines or reports them; use it to make data human-readable or to prepare reports. Quick demonstration Create a sample file, observe adjacent duplicates, then try counting; example commands and outputs are shown here: ```$ cat lines.txt apple apple banana Banana banana carrot carrot $ uniq lines.txt apple banana Banana banana carrot $ sort lines.txt | uniq -c 1 Banana 2 apple 2 banana 2 carrot $ sort lines.txt | uniq -i -c 3 banana 2 apple 2 carrot ``` Options that change behavior Use `-c` to prefix counts, `-d` to show only repeated lines, and `-u` to show only non-repeated lines; `-i` ignores case, `-f` skips fields and `-s` skips characters when comparing, `-w` limits comparison width, and `-z` handles NUL-delimited input; remember that `uniq` only compares adjacent lines, so pipe through `sort` for global deduplication or use `awk` for more complex grouping. When to reach for other tools For line counting and aggregation consider `sort | uniq -c` as the canonical pair, use `awk` for field-aware totals, and prefer `sed` or `perl` for pattern-driven collapses when comparison rules exceed `uniq`'s simple options. Takeaway and next steps Mastering `uniq` is about knowing its adjacency rule and combining it wisely; practice with logs and CSVs, then expand into `awk` and `sed` for richer processing — and consider formalizing skills with CompTIA Linux+ or LPIC-1, using intensive preparation at bitsandbytes.academy. Join Bits & Bytes Academy First class LINUX exam preparation. utilities scripting filesystem
