Finding things
xargs
Read items from standard input and run a command with them as arguments, in batches or in parallel.
xargs [OPTION...] [COMMAND [INITIAL-ARGUMENT...]]
Many commands take their input as arguments (rm a b c), not on standard input. xargs
bridges the gap: it reads items from stdin and appends them to a command line, packing in as
many as the system allows and running the command as few times as possible. With no command it
runs echo.
By default xargs splits its input on any whitespace and also interprets quotes and
backslashes, which is exactly wrong for file names with spaces. The robust form is NUL
separation end to end: find -print0, grep -lZ or printf '%s\0' on the left, xargs -0
on the right. GNU xargs also has -d '\n' to split on newlines only.
The options you will use: -n N (at most N items per command), -I {} (run once per line and
put the item where {} appears), -P N (run N commands in parallel), -r (do nothing on
empty input) and -t (print each command before running it). When the items come from
find, find -exec ... {} + does the same job without a pipe.
Examples
Turn lines into arguments
printf '%s\n' web01 web02 db01 | xargs echo ping
ping web01 web02 db01
xargs reads the three lines and runs echo ping web01 web02 db01 once. All items go on one
command line unless you limit them.
Join words onto one line
xargs < todo.txt
buy milk fix login bug write tests fix logout bug deploy v2.4
With no command, xargs runs echo. Every whitespace-separated word of the file becomes an
argument, so the five lines collapse into one. This also shows the default splitting: a
line break and a space are the same thing to xargs.
Limit the number of arguments per command with -n
xargs -n 2 < data/hosts.txt
web01 10.0.0.11 web02 10.0.0.12 db01 10.0.0.21 cache01 10.0.0.31
-n 2 passes at most two items per run, so echo runs four times, pairing each host name
with its address.
Show each command before it runs with -t
seq 5 | xargs -t -n 2 echo
echo 1 2 1 2 echo 3 4 3 4 echo 5 5
-t writes each command line to stderr before running it. It is the quickest way to see
how xargs is batching your input. The last batch has only one item left.
Put the item in the middle of the command with -I
cut -d' ' -f1 data/hosts.txt | xargs -I{} echo "checking {} on port 22"
checking web01 on port 22 checking web02 on port 22 checking db01 on port 22 checking cache01 on port 22
-I{} runs the command once per input line and substitutes the line wherever {}
appears. Leading blanks are stripped, but spaces inside the line are kept, so each line
stays one item.
Why plain xargs breaks on spaces
ls *.txt | xargs wc -l
wc: file: No such file or directory wc: with: No such file or directory wc: spaces.txt: No such file or directory 6 notes.txt 5 todo.txt 11 total
file with spaces.txt is split into three arguments, none of which exists, so wc fails
on them. When the command fails for any batch, xargs exits with status 123.
Handle any file name with -0
printf '%s\0' *.txt | xargs -0 wc -l
1 file with spaces.txt 6 notes.txt 5 todo.txt 12 total
With -0 items are separated by NUL bytes, which cannot occur in file names, and quotes
and backslashes lose any special meaning. The file with spaces arrives as one argument.
Combine with find -print0
find src -name '*.py' -print0 | xargs -0 grep -c TODO
src/lib/helpers.py:1 src/app.py:1
The standard safe pipeline: -print0 on the find side, -0 on the xargs side. grep gets
both files in one run, so it prints a file:count line for each.
Split on newlines only with -d
printf 'file with spaces.txt\ntodo.txt\n' | xargs -d '\n' wc -c
34 file with spaces.txt 62 todo.txt 96 total
-d '\n' (GNU) makes newline the only separator and turns off quote processing, so names
with spaces survive when the input is one name per line. It still breaks on names that
contain newlines; only -0 handles those.
Do nothing on empty input with -r
grep -l CRITICAL logs/*.log | xargs echo "would delete:"
grep -l CRITICAL logs/*.log | xargs -r echo "would delete:"
echo "exit status: $?"
would delete: exit status: 0
No file matches, so xargs gets no input. GNU xargs still runs the command once by default,
which here prints a bare would delete: and with rm would print a usage error. -r
(--no-run-if-empty) skips the run, so the second line prints nothing and the status is 0.
Run commands in parallel with -P
time (seq 4 | xargs -P 4 -I{} sh -c 'sleep 1; echo "job {} done"')
job 1 done job 2 done job 3 done job 4 done real 0m1.006s user 0m0.005s sys 0m0.008s
-P 4 keeps up to four commands running at once. Four one-second jobs finish in about one
second instead of four. Completion order is whatever order the jobs finish in, so it can
differ between runs. -P 0 runs as many as possible.
Run several commands per item
find data -name '*.csv' -print0 | xargs -0 sh -c 'for f; do printf "%s: %s rows\n" "$f" "$(($(wc -l < "$f") - 1))"; done' sh
data/users.csv: 8 rows data/sales.csv: 12 rows
When each item needs a small script, hand the batch to sh -c. The trailing sh fills
$0, so the file names become $1, $2, ... and for f loops over them. This avoids
-I{} inside the script text, where a file name containing quotes could break out of it.
Edit every file that contains a string
grep -rlZ 'db01' data | xargs -0 sed -i 's/db01/db02/g'
grep -rn 'db0[12]' data
data/hosts.txt:3:db02 10.0.0.21
data/servers.json:4: {"name": "db02", "ip": "10.0.0.21", "role": "database", "cpus": 16, "tags": ["prod"], "up": true},
data/config.json:6: "database": {"host": "db02", "port": 5432, "name": "demo", "pool": 10}, grep -lZ lists the matching files NUL-terminated, and sed -i edits all three in one
run. The check afterwards searches for either name and finds only db02 left. On real
files, run the grep -rl alone first to see what will be changed.
Common pitfalls
Plain
xargssplits on spaces and treats quotes and backslashes specially, so a file namedit's.txtmakes it fail with an unmatched quote error. Use-0with NUL-separated input.GNU xargs runs the command once even on empty input, and
rmwith no arguments then fails. Add-r.-I{}implies one item per line and one command per item, so it is much slower than batching on large inputs.Never splice
{}into ash -cstring (xargs -I{} sh -c 'echo {}'): a crafted file name becomes shell code. Pass items as arguments and use"$1"instead.With
-P, outputs from parallel commands can interleave in the middle of a line. Have each job write to its own file, or use a tool built for parallel jobs, when order matters.
GNU vs BSD and macOS
The examples above ran the GNU/Linux version. These are the differences that matter on macOS and the BSDs (written from their documentation, not run here).
BSD xargs, including the one on macOS, already skips the command on empty input, so
-rmatters only for GNU. FreeBSD accepts-ras a no-op for compatibility.-dis GNU only. On macOS convert newlines to NULs first:tr '\n' '\0' | xargs -0.macOS xargs uses
-Iand-Jfor replacement strings; GNU's older-ispelling is deprecated and not portable.
Options worth knowing
A selection, described in our own words. For the full list, see xargs(1) or xargs --help.
| Option | What it does |
|---|---|
-0, --null | Items are separated by NUL bytes; quotes and backslashes are not special. |
-d, --delimiter DELIM | Items are separated by the character DELIM (GNU). |
-n, --max-args N | Use at most N items per command line. |
-L, --max-lines N | Use at most N input lines per command line. |
-I REPLACE | Run once per line, replacing REPLACE in the arguments with the line. |
-P, --max-procs N | Run up to N commands at the same time (0 means as many as possible). |
-r, --no-run-if-empty | Do not run the command if the input is empty. |
-t, --verbose | Print each command line to stderr before running it. |
-p, --interactive | Ask before running each command. |
-s, --max-chars N | Limit each command line to N characters. |
-a, --arg-file FILE | Read items from FILE instead of standard input. |
-x, --exit | Exit if a command line would exceed the size limit. |
--show-limits | Print the command-line length limits in effect. |
Related
findWalk a directory tree and print, or act on, every file that matches a set of tests.grepPrint the lines of files (or standard input) that match a pattern.sedEdit a stream of text with a script of commands, most often to substitute text or delete lines.printfPrint text and values through a format string, with precise control over spacing, numbers and escapes.echoPrint its arguments separated by spaces, followed by a newline.- Pipes and pipelinesHow cmd1 | cmd2 connects processes, which exit status a pipeline returns, PIPESTATUS, pipefail, SIGPIPE, and why variables set in a piped loop vanish.
- QuotingSingle quotes, double quotes, backslashes and $'...', what each protects, and the unquoted-variable bugs they prevent, each shown failing and then fixed.
- Word splittingWhen bash splits expansion results into separate words, how IFS controls it, and why `for line in $(cat file)` does not loop over lines.