Uniq


uniq is a utility command on Unix and Unix-like operating systems which, when fed a text file or STDIN, outputs the text with adjacent identical lines collapsed to one, unique line of text.

Overview

The command is a kind of filter program. Typically it is used after sort. It can also output only the duplicate lines, or add the number of occurrences of each line. For example, the following command lists the unique lines in a file, sorted by the number of times each occurs:

$ sort file | uniq -c | sort -n

Using uniq like this is common when building pipelines in shell scripts.

History

First appearing in Version 3 Unix, uniq is now available for a number of different Unix and Unix-like operating systems. It is part of the X/Open Portability Guide since issue 2 of 1987. It was inherited into the first version of POSIX and the Single Unix Specification.
The version bundled in GNU coreutils was written by Richard Stallman and David MacKenzie.
The command is available as a separate package for Microsoft Windows as part of the GnuWin32 project and the UnxUtils collection of native Win32 ports of common GNU Unix-like utilities.
A uniq command is also part of ASCII's MSX-DOS2 Tools for MSX-DOS version 2.