Archives and compression
gzip
Compress files to .gz (replacing the original by default), or decompress them with -d or gunzip.
gzip [OPTION...] [FILE...]
gzip FILE compresses one file into FILE.gz and deletes the original, keeping its name,
permissions and modification time. gzip -d FILE.gz (or gunzip) reverses it. Each file is
compressed separately: gzip has no notion of a directory archive, which is why directories are
bundled with tar first and the result is a .tar.gz.
With no file, or with -c, gzip works as a filter between standard input and output, so it
slots into pipelines: pg_dump db | gzip > db.sql.gz. To read a compressed file without
unpacking it, use zcat (and its relatives zgrep, zless, zdiff).
gzip is fast and readable everywhere, which is why logs, tarballs and HTTP responses use it.
zstd compresses about as well while being much faster, and xz compresses better but
slower; reach for those when you control both ends.
Examples
Compress a file in place
gzip access.log
ls -l access.log*
-rw-rw-r-- 1 dev dev 591 Oct 24 21:55 access.log.gz
access.log is gone, replaced by access.log.gz at about a quarter of the size. The date
is the original file's modification time, which gzip carries over.
Keep the original with -k
gzip -k access.log
ls -l access.log*
-rw-rw-r-- 1 dev dev 2309 Oct 24 21:55 access.log -rw-rw-r-- 1 dev dev 591 Oct 24 21:55 access.log.gz
-k (--keep) leaves the input file in place, so both versions exist side by side.
Decompress with -d or gunzip
Setup run before this example
gzip notes.txtls notes*
gzip -d notes.txt.gz
ls notes*
notes.txt.gz notes.txt
gzip -d and gunzip are the same thing. The .gz file is replaced by the original
notes.txt.
Write to standard output with -c
gzip -c todo.txt > todo.txt.gz
ls todo*
todo.txt todo.txt.gz
-c leaves the input alone and sends the compressed bytes to standard output, so you choose
the output name with a redirection. It is also how you compress into another directory.
Show compressed and original sizes
Setup run before this example
gzip -k access.loggzip -l access.log.gz
compressed uncompressed ratio uncompressed_name
591 2309 75.7% access.log -l reads the sizes from the file's trailer without decompressing it. The ratio is the
space saved: 75.7% here. The stored size is modulo 4 GiB, so it is wrong for larger files.
Report the saving per file
gzip -v data/sales.csv data/users.csv
data/sales.csv: 61.7% -- replaced with data/sales.csv.gz data/users.csv: 43.6% -- replaced with data/users.csv.gz
-v prints, for each file, the percentage saved and what it was replaced with.
Trade speed for size with -1 to -9
gzip -1 -c access.log | wc -c
gzip -9 -c access.log | wc -c
658 591
-1 is fastest, -9 compresses best, and the default is -6. Here -9 saves about 10%
over -1; on large files it also costs noticeably more time, for a few percent over the
default.
Compress the output of a command
grep ' 404 ' access.log | gzip > not-found.log.gz
zcat not-found.log.gz | wc -l
4
With no file name gzip reads standard input and writes standard output. zcat reads it
back to check: the four matching lines survived the trip.
Compress every file in a directory tree
gzip -r logs
ls logs
app.log.1.gz app.log.gz error.log.gz
-r walks into the directory and compresses each file separately. The directory itself is
not archived: for one file containing a whole tree, use tar -czf.
Concatenated gzip files are still valid
gzip -c todo.txt > all.gz
gzip -c data/poem.txt >> all.gz
zcat all.gz
buy milk fix login bug write tests fix logout bug deploy v2.4 The quick brown fox jumps over the lazy dog. The dog sleeps; the fox runs on.
A gzip file may hold several compressed members one after another, and decompression joins
them. That is why appending to a .gz log with >> works.
Check that a file is intact
Setup run before this example
gzip -k access.log
head -c 300 access.log.gz > cut.gzgzip -t access.log.gz && echo "access.log.gz is fine"
gzip -t cut.gz
access.log.gz is fine gzip: cut.gz: unexpected end of file
-t decompresses in memory and checks the CRC without writing anything. The truncated copy
fails with unexpected end of file and exit status 1.
gzip will not overwrite an existing file
Setup run before this example
gzip -k notes.txtgzip -d notes.txt.gz
gzip: notes.txt already exists; not overwritten
notes.txt is still there (the .gz was made with -k). In a script, gzip refuses with
exit status 2 instead of clobbering it; at a terminal it asks first. -f forces the
overwrite.
Common pitfalls
Plain
gzip filedeletesfile. Use-kor-cwhen you need to keep the original.gzip dirdoes nothing useful (it skips directories with a warning), andgzip -r dirproduces a pile of separate.gzfiles. Archive a directory withtar -czf dir.tar.gz dir.gzip -lgets the original size from a 32-bit field, so it shows the size modulo 4 GiB. For big files,zcat file.gz | wc -cis the reliable count.gzip -c a b > out.gzproduces one file with two members; decompressing givesaandbjoined, not two files.
GNU vs BSD and macOS
The examples above ran the GNU/Linux version. These are the differences that matter on macOS and the BSDs (written from their documentation, not run here).
macOS and FreeBSD ship their own gzip with the same everyday options (
-c -d -k -l -r -t -v -1..-9). Older versions lacked-k;gzip -c f > f.gzworks everywhere.
Options worth knowing
A selection, described in our own words. For the full list, see gzip(1) or gzip --help.
| Option | What it does |
|---|---|
-c, --stdout | Write to standard output and keep the input files. |
-d, --decompress | Decompress instead of compress. |
-k, --keep | Keep the input files instead of deleting them. |
-f, --force | Overwrite existing output files and compress links or already-compressed names. |
-l, --list | Show compressed size, original size and ratio of .gz files. |
-r, --recursive | Compress or decompress every file inside the named directories. |
-t, --test | Check the integrity of compressed files without writing output. |
-v, --verbose | Print the name and saving for each file. |
-q, --quiet | Suppress warnings. |
-n, --no-name | Do not store the original name and timestamp (when compressing). |
-N, --name | Store or restore the original name and timestamp. |
-S, --suffix SUF | Use SUF instead of .gz. |
-1, --fast | Compress fastest, with the least compression. |
-9, --best | Compress best, most slowly. |
--rsyncable | Make output that rsync can transfer efficiently after small changes. |
Related
zcatPrint the decompressed contents of gzip files to standard output, leaving the files compressed.tarBundle files and directories into one archive, list it, or extract it; optionally compressed with gzip, bzip2, xz or zstd.bzip2Compress files to .bz2 with the Burrows-Wheeler algorithm, or decompress them with -d, bunzip2 or bzcat.xzCompress files to .xz with LZMA2 for high ratios, or decompress them with -d, unxz or xzcat.zstdCompress files to .zst with Zstandard, fast at any ratio; decompress with -d, unzstd or zstdcat.zipPackage and compress files into a .zip archive, the format every operating system opens natively.