Text processing cheat sheet

The most useful text processing commands on one printable page, every line checked by running it.

`awk` Scan text as records and fields, and run a small program on each line that matches a pattern.

$ awk '{print $1}' data/hosts.txt: Print one column output
$ awk -F, '{print $2 " <" $3 ">"}' data/users.csv: Split on a comma with -F output
$ awk -F, '$4 >= 10' data/sales.csv: Print the lines where a field meets a condition output
$ awk -F, 'NR > 1 { units += $4; revenue += $4 * $5 } END { print units, revenue }' data/sales.csv: Total a column output
$ awk '{count[$1]++} END {for (ip in count) print count[ip], ip}' access.log | sort -rn: Count occurrences with an associative array output

`base64` Encode binary data as base64 text, or decode base64 back to the original bytes with -d.

$ printf 'hello' | base64: Encode a string output
$ echo 'aGVsbG8sIHdvcmxkCg==' | base64 -d: Decode base64 output
$ base64 -w 0 todo.txt; echo: One long line without wrapping output

`bc` An arbitrary-precision calculator language, used from scripts for decimal math the shell cannot do.

$ echo '10 / 3' | bc > echo 'scale=2; 10 / 3' | bc: Division is integer until you set scale output
$ echo '10 / 3; sqrt(2); 4 * a(1)' | bc -l: Use the math library with -l output
$ load=3.75; limit=2.5 > if (( $(echo "$load > $limit" | bc) )); then echo "load $load is over $limit"; fi: Compare decimal numbers output
$ echo 'obase=16; 255' | bc > echo 'obase=2; 10' | bc > echo 'ibase=16; FF' | bc: Convert between number bases output

`cmp` Compare two files byte by byte and report the first difference, or just whether they differ.

$ sed 's/fix logout/fix signup/' todo.txt > todo2.txt > cmp todo.txt todo2.txt: Find the first difference between two files output
$ if cmp -s photos/beach.png photos/forest.png; then echo same; else echo "different (status $?)"; fi: Test for equality in a script with -s output

`column` Format input into aligned columns or a table; turns CSV and whitespace data into readable output.

$ column -t -s , data/users.csv | head -n 5: Read a CSV as an aligned table output
$ column -t data/hosts.txt: Align whitespace-separated data output
$ column -t -s $'\t' -R 2,3 data/inventory.tsv: Right-align numeric columns output

`comm` Compare two sorted files line by line and show lines unique to each and lines in both.

$ comm -12 expected.txt running.txt: Lines in both files output
$ comm -23 expected.txt running.txt: Lines only in the first file output
$ comm -3 <(sort inventory.txt) <(cut -d ' ' -f 1 data/hosts.txt | sort): Compare unsorted lists output

`cut` Print selected fields, characters or bytes from each line, such as columns of a CSV or TSV file.

$ cut -d , -f 2,4 data/users.csv | head -n 4: Select columns of a CSV output
$ cut -f 2 data/inventory.tsv: Tab is the default delimiter output
$ cut -d , -f 3- data/sales.csv | head -n 3: A range of fields to the end of the line output
$ cut -d , --complement -f 3 data/users.csv | head -n 3: Everything except some fields output

`diff` Compare two files or directories line by line and print the changes that turn one into the other.

$ diff -u old.conf new.conf: Show a unified diff with context output
$ cp todo.txt todo.copy > diff -q todo.txt todo.copy && echo "same" > echo 'call the bank' >> todo.copy > diff -q todo.txt todo.copy || echo "exit status: $?": Use the exit status to test for changes output
$ sed 's/ = /=/; /^$/d' data/settings.conf > reformatted.conf > diff -q data/settings.conf reformatted.conf > diff -w -B data/settings.conf reformatted.conf && echo "only whitespace changed": Ignore whitespace and blank-line changes output
$ diff -r src.orig src: Compare two directories output

`echo` Print its arguments separated by spaces, followed by a newline.

$ echo Hello, world: Print a message output
$ echo -n "no newline here"; echo " <- same line": Leave off the trailing newline with -n output
$ echo 'col1\tcol2' > echo -e 'col1\tcol2\nnext line': Interpret escapes with -e in bash output
$ echo 'col1\tcol2' > echo -e 'col1\tcol2': The same commands under sh behave differently output

`envsubst` Replace $VAR and ${VAR} references in text with the values of environment variables.

$ export PORT=8080 DOMAIN=demo.example APP=demo > envsubst < app.conf.tpl > app.conf > cat app.conf: Render a config file from a template output
$ export PORT=8080 > envsubst '$PORT' < site.tpl: Substitute only selected variables output

`expand` Convert tabs to spaces, keeping the columns where they were; unexpand does the reverse.

$ expand data/inventory.tsv | cat -A: Replace tabs with spaces output
$ printf 'if ok:\n\tif ready:\n\t\treturn 1\n' | expand -t 4: Tab stops every 4 columns output

`expr` Evaluate an integer, comparison or string expression given as separate arguments and print the result.

$ expr 7 + 5 > expr 7 / 2 > expr 7 % 2: Do integer arithmetic output
$ expr 5 - 5; echo "exit status: $?" > expr 5 - 4; echo "exit status: $?": A result of zero sets exit status 1 output
$ file='backup-20260314.tar.gz' > expr "$file" : 'backup-$[0-9]*$' > expr "$file" : '.*\.$.*$': Extract part of a string with a group output

`fmt` Reflow text into paragraphs of a given width, joining short lines and wrapping long ones.

$ printf 'This line is short.\nThis one is a little longer than the first one, and it keeps going for a while.\nEnd.\n' | fmt -w 40: Rewrap a paragraph to a width output
$ fmt -s -w 20 notes.txt: Wrap long lines without joining short ones output

`fold` Wrap long lines to a fixed width, optionally breaking only at spaces.

$ fold -w 20 README.md: Wrap at a fixed width output
$ fold -s -w 20 notes.txt: Break at spaces instead of inside words output

`grep` Print the lines of files (or standard input) that match a pattern.

$ grep ERROR logs/app.log: Find lines containing a word output
$ grep -v -e '^#' -e '^$' data/settings.conf: Show the lines that do not match output
$ grep -rn TODO .: Search a directory tree output
$ grep -oE '[0-9]{1,3}(\.[0-9]{1,3}){3}' logs/app.log: Print only the matched text output
$ grep -n -B1 -A2 'health check failed' logs/app.log: Show lines around each match output

`iconv` Convert text from one character encoding to another, such as Latin-1 or UTF-16 to UTF-8.

$ file legacy.txt > iconv -f ISO-8859-1 -t UTF-8 legacy.txt: Convert a Latin-1 file to UTF-8 output
$ echo 'Grüße – “naïve” café' | iconv -f UTF-8 -t ASCII//TRANSLIT: Approximate missing characters output
$ iconv -f UTF-8 -t UTF-8 todo.txt > /dev/null && echo "todo.txt: valid UTF-8" > iconv -f UTF-8 -t UTF-8 legacy.txt > /dev/null || echo "legacy.txt: not UTF-8": Check that a file is valid UTF-8 output

`join` Join lines of two sorted files that share a key field, like a database join on text files.

$ join <(sort data/hosts.txt) roles.txt: Join two files on their first field output
$ join -t , -1 4 -2 1 <(tail -n +2 data/users.csv | sort -t , -k 4,4) floors.csv: Join CSV files on a column other than the first output
$ join -t , -1 4 -2 1 -a 1 -e 'no desk' -o 1.2,2.2 <(tail -n +2 data/users.csv | sort -t , -k 4,4) floors.csv: Keep lines without a match (outer join) output

`jq` Query, filter and reshape JSON from the command line with a small expression language.

$ jq -r '.database.host' data/config.json: Print strings without quotes output
$ jq -r '.[].name' data/servers.json: Iterate over an array output
$ jq -r '.[] | select(.cpus >= 8) | .name' data/servers.json: Filter with select output
$ jq -c '.[] | {host: .name, addr: "\(.ip):22", prod: (.tags | contains(["prod"]))}' data/servers.json: Build new objects output
$ role=web > jq -r --arg role "$role" '.[] | select(.role == $role) | "\(.name) \(.ip)"' data/servers.json: Use a shell variable safely with --arg output

`md5sum` Compute or verify MD5 checksums; fine for spotting accidental changes, not for security.

$ md5sum todo.txt: Hash a file output
$ md5sum logs/*.log > logs.md5 > md5sum -c logs.md5: Save and verify checksums output

`paste` Merge lines of files side by side, or join all lines of one file into a single line.

$ paste data/hosts.txt data/numbers.txt | head -n 4: Put two files next to each other output
$ paste -s -d , todo.txt: Join all lines into one, comma-separated output
$ paste -s -d + data/numbers.txt | bc: Add up a column of numbers output
$ paste - - < data/fruits.txt: Turn a list into rows of two output

`patch` Apply a diff to files, turning the original versions into the changed ones.

$ patch < todo.patch > cat todo.txt: Apply a patch to one file output
$ patch --dry-run < todo.patch > cat todo.txt: Check a patch before applying it output
$ patch -p1 < fix.patch > grep printf src/main.c: Strip path prefixes with -p1 output
$ patch < todo.patch > patch -R < todo.patch > sed -n 3p todo.txt: Undo a patch with -R output

`printf` Print text and values through a format string, with precise control over spacing, numbers and escapes.

$ printf 'Hello, %s. You have %d new messages.\n' "$USER" 3: Print a value inside a sentence output
$ printf '%s\n' apple 'cherry pie' banana: Reuse the format for every argument output
$ printf '%-10s|%6s|%-6s|\n' name qty price > printf '%-10s|%6d|%6.2f|\n' widget 12 4.5 gadget 3 19.99: Align columns with widths output
$ msg='Disk 100% done' > printf "$msg\n" > printf '%s\n' "$msg": Never put data in the format string output
$ printf '%q\n' 'file with spaces.txt' "it's" $'tab\there': Quote a string for reuse in the shell with %q output

`rev` Reverse the characters of every line.

$ rev todo.txt: Reverse each line output
$ echo /var/log/demo/app.log | rev | cut -d / -f 1 | rev: Take the last field of each line output

`sed` Edit a stream of text with a script of commands, most often to substitute text or delete lines.

$ sed 's/fix/FIX/' todo.txt: Replace the first match on each line output
$ echo 'a-b-c-d' | sed 's/-/:/' > echo 'a-b-c-d' | sed 's/-/:/g' > echo 'a-b-c-d' | sed 's/-/:/2': Replace every match on the line output
$ sed -i.bak 's/^port = .*/port = 9090/' data/settings.conf > grep -H '^port' data/settings.conf data/settings.conf.bak: Edit a file in place and keep a backup output
$ sed -e '/^#/d' -e '/^$/d' data/settings.conf: Delete comments and blank lines output
$ sed -n '3,5p' logs/app.log: Print a range of lines by number output

`seq` Print a sequence of numbers, with a chosen start, step, separator and format.

$ seq 5: Count from 1 to N output
$ seq 2 2 10 | paste -s -d ' ': Choose the start and the step output
$ seq -w 8 11 | paste -s -d ' ': Pad with zeros to equal width output
$ seq -f 'backup-%03g.tar' 3: Generate file names output

`sha256sum` Compute SHA-256 checksums of files, or verify files against a list of checksums with -c.

$ sha256sum todo.txt: Hash a file output
$ sha256sum data/*.csv > SHA256SUMS > cat SHA256SUMS: Save checksums to a file output
$ sha256sum data/*.csv > SHA256SUMS > sha256sum -c SHA256SUMS: Verify files against a checksum list output
$ expected=328f70102b0d08bbdea3cbb5b0e70339167d51a8260a023be34668b70f303ffe > echo "$expected todo.txt" | sha256sum -c: Check a download against a published hash output

`shuf` Shuffle lines into random order, or pick random lines, numbers or words.

$ shuf todo.txt: Shuffle all lines output
$ shuf -n 1 data/fruits.txt: A random line from a file output
$ shuf -i 1-6 -n 1: A random number in a range output

`sort` Sort lines of text alphabetically, numerically, by version or by chosen fields; also merge and de-duplicate.

$ sort data/numbers.txt | paste -s -d ' ' > sort -n data/numbers.txt | paste -s -d ' ': Sort numbers by value output
$ sort -u data/fruits.txt: Sort and remove duplicates output
$ tail -n +2 data/sales.csv | sort -t , -k 4,4n | head -n 4: Sort a CSV by one column output
$ printf '10K\n2M\n512\n1G\n3K\n' | sort -h: Sort human-readable sizes output
$ printf 'v2.10\nv2.9\nv2.4.1\nv10\n' | sort -V: Sort version numbers output

`split` Split a file into pieces by number of lines, by size or into N equal parts.

$ seq 1000 > numbers.txt > split -l 300 numbers.txt > wc -l x*: Split by number of lines output
$ seq 1000 > numbers.txt > split -l 400 -d -a 3 numbers.txt part. > ls part.*: Choose a prefix and numeric suffixes output
$ head -c 2500 /dev/urandom > blob.bin > split -b 1K blob.bin blob.part. > ls -l blob.part.*: Split a binary file by size output
$ head -c 5000 /dev/urandom > blob.bin > split -b 2K blob.bin piece. > cat piece.* > restored.bin > cmp blob.bin restored.bin && echo identical: Put the pieces back together output

`tee` Copy standard input to standard output and to one or more files at the same time.

$ grep ERROR logs/app.log | tee errors.txt > echo "--- saved:"; wc -l errors.txt: See output and save it to a file output
$ echo "first run" | tee run.log > echo "second run" | tee -a run.log > echo "---"; cat run.log: Append instead of overwriting output
$ printf 'API_URL=http://localhost:9090\nDEBUG=true\n' | tee .env.local > /dev/null > cat .env.local: Write a file without printing anything output

`tr` Translate, squeeze or delete characters from standard input; for case changes, line endings and stripping characters.

$ tr a-z A-Z < todo.txt: Convert to upper case output
$ tr -d '\r' < data/windows.txt | od -c: Remove Windows carriage returns output
$ echo 'alpha beta gamma' | tr -s ' ': Squeeze repeated spaces output
$ echo 'Phone: (555) 123-4567' | tr -cd '0-9'; echo: Keep only certain characters output

`uniq` Collapse or report repeated adjacent lines; with sort, count how often each line occurs.

$ sort data/fruits.txt | uniq: Remove all duplicates output
$ sort data/fruits.txt | uniq -c: Count occurrences output
$ cut -d ' ' -f 3 logs/app.log | sort | uniq -c | sort -rn: Most frequent first output
$ sort data/fruits.txt | uniq -d: Show only the repeated lines output

From https://opscape.vip/cheatsheets/text/ · checked 2026-10-31

awk Scan text as records and fields, and run a small program on each line that matches a pattern.

base64 Encode binary data as base64 text, or decode base64 back to the original bytes with -d.

bc An arbitrary-precision calculator language, used from scripts for decimal math the shell cannot do.

cmp Compare two files byte by byte and report the first difference, or just whether they differ.

column Format input into aligned columns or a table; turns CSV and whitespace data into readable output.

comm Compare two sorted files line by line and show lines unique to each and lines in both.

cut Print selected fields, characters or bytes from each line, such as columns of a CSV or TSV file.

diff Compare two files or directories line by line and print the changes that turn one into the other.

echo Print its arguments separated by spaces, followed by a newline.

envsubst Replace $VAR and ${VAR} references in text with the values of environment variables.

expand Convert tabs to spaces, keeping the columns where they were; unexpand does the reverse.

expr Evaluate an integer, comparison or string expression given as separate arguments and print the result.

fmt Reflow text into paragraphs of a given width, joining short lines and wrapping long ones.

fold Wrap long lines to a fixed width, optionally breaking only at spaces.

grep Print the lines of files (or standard input) that match a pattern.

iconv Convert text from one character encoding to another, such as Latin-1 or UTF-16 to UTF-8.

join Join lines of two sorted files that share a key field, like a database join on text files.

jq Query, filter and reshape JSON from the command line with a small expression language.

md5sum Compute or verify MD5 checksums; fine for spotting accidental changes, not for security.

paste Merge lines of files side by side, or join all lines of one file into a single line.

patch Apply a diff to files, turning the original versions into the changed ones.

printf Print text and values through a format string, with precise control over spacing, numbers and escapes.

rev Reverse the characters of every line.

sed Edit a stream of text with a script of commands, most often to substitute text or delete lines.

seq Print a sequence of numbers, with a chosen start, step, separator and format.

sha256sum Compute SHA-256 checksums of files, or verify files against a list of checksums with -c.

shuf Shuffle lines into random order, or pick random lines, numbers or words.

sort Sort lines of text alphabetically, numerically, by version or by chosen fields; also merge and de-duplicate.

split Split a file into pieces by number of lines, by size or into N equal parts.

tee Copy standard input to standard output and to one or more files at the same time.

tr Translate, squeeze or delete characters from standard input; for case changes, line endings and stripping characters.

uniq Collapse or report repeated adjacent lines; with sort, count how often each line occurs.

`awk` Scan text as records and fields, and run a small program on each line that matches a pattern.

`base64` Encode binary data as base64 text, or decode base64 back to the original bytes with -d.

`bc` An arbitrary-precision calculator language, used from scripts for decimal math the shell cannot do.

`cmp` Compare two files byte by byte and report the first difference, or just whether they differ.

`column` Format input into aligned columns or a table; turns CSV and whitespace data into readable output.

`comm` Compare two sorted files line by line and show lines unique to each and lines in both.

`cut` Print selected fields, characters or bytes from each line, such as columns of a CSV or TSV file.

`diff` Compare two files or directories line by line and print the changes that turn one into the other.

`echo` Print its arguments separated by spaces, followed by a newline.

`envsubst` Replace $VAR and ${VAR} references in text with the values of environment variables.

`expand` Convert tabs to spaces, keeping the columns where they were; unexpand does the reverse.

`expr` Evaluate an integer, comparison or string expression given as separate arguments and print the result.

`fmt` Reflow text into paragraphs of a given width, joining short lines and wrapping long ones.

`fold` Wrap long lines to a fixed width, optionally breaking only at spaces.

`grep` Print the lines of files (or standard input) that match a pattern.

`iconv` Convert text from one character encoding to another, such as Latin-1 or UTF-16 to UTF-8.

`join` Join lines of two sorted files that share a key field, like a database join on text files.

`jq` Query, filter and reshape JSON from the command line with a small expression language.

`md5sum` Compute or verify MD5 checksums; fine for spotting accidental changes, not for security.

`paste` Merge lines of files side by side, or join all lines of one file into a single line.

`patch` Apply a diff to files, turning the original versions into the changed ones.

`printf` Print text and values through a format string, with precise control over spacing, numbers and escapes.

`rev` Reverse the characters of every line.

`sed` Edit a stream of text with a script of commands, most often to substitute text or delete lines.

`seq` Print a sequence of numbers, with a chosen start, step, separator and format.

`sha256sum` Compute SHA-256 checksums of files, or verify files against a list of checksums with -c.

`shuf` Shuffle lines into random order, or pick random lines, numbers or words.

`sort` Sort lines of text alphabetically, numerically, by version or by chosen fields; also merge and de-duplicate.

`split` Split a file into pieces by number of lines, by size or into N equal parts.

`tee` Copy standard input to standard output and to one or more files at the same time.

`tr` Translate, squeeze or delete characters from standard input; for case changes, line endings and stripping characters.

`uniq` Collapse or report repeated adjacent lines; with sort, count how often each line occurs.