opscape

Bash scripting guide · part 21 of 32

Redirection

How >, >>, 2>, 2>&1, &>, < and exec move a command's input and output between files and file descriptors, and why the order matters.

Every process starts with three open file descriptors: 0 is standard input, 1 is standard output and 2 is standard error. A redirection tells the shell to open, close or copy one of those descriptors before the command runs. The command itself never sees the > or the file name; it just writes to fd 1 as usual and the bytes land wherever the shell pointed it.

That last point explains most of the rules below. Redirections are processed by the shell, left to right, before the command starts. Each one is an assignment to a descriptor, not a pipe or a stream that "flows" somewhere.

Output to a file: > and >>

Write standard output to a file

echo "first run" > out.txt
echo "second run" > out.txt
cat out.txt
output
second run

> opens the file for writing and truncates it to zero length first, so the second command replaced the first line instead of adding to it. The file is created if it does not exist.

Append instead of overwrite

echo "first run" >> out.txt
echo "second run" >> out.txt
cat out.txt
output
first run
second run

>> opens the file in append mode: every write goes to the current end of the file, so both lines survive. This is the mode to use for log files that several processes write to.

Standard error: 2>

Error messages go to fd 2 precisely so that they do not end up mixed into data you redirected. Put the descriptor number directly in front of the operator, with no space: 2> redirects fd 2. (2 > file would pass 2 as an argument and redirect fd 1.)

Send error messages to their own file

ls todo.txt missing.txt > found.txt 2> errors.txt
echo "ls exited with $?"
echo "--- found.txt"; cat found.txt
echo "--- errors.txt"; cat errors.txt
output
ls exited with 2
--- found.txt
todo.txt
--- errors.txt
ls: cannot access 'missing.txt': No such file or directory

ls found one file and not the other. The name it found went to fd 1 and therefore into found.txt; the complaint went to fd 2 and into errors.txt. The exit status 2 is ls's way of saying something went wrong, independent of where the messages ended up.

Merging the two streams: 2>&1 and why order matters

2>&1 means "make fd 2 a copy of whatever fd 1 is right now". It duplicates the descriptor at that moment; it does not create a lasting link. So the position of 2>&1 relative to > file changes the result.

The order of > and 2>&1

echo "== 2>&1 > all.txt"
ls todo.txt missing.txt 2>&1 > all.txt
echo "== > all.txt 2>&1"
ls todo.txt missing.txt > all.txt 2>&1
echo "== all.txt now contains:"
cat all.txt
output
== 2>&1 > all.txt
ls: cannot access 'missing.txt': No such file or directory
== > all.txt 2>&1
== all.txt now contains:
ls: cannot access 'missing.txt': No such file or directory
todo.txt

In the first form, 2>&1 copies fd 1 while it still points at the terminal, and only then does > all.txt move fd 1. The error therefore still appears on screen. In the second form fd 1 is moved first, and fd 2 copies the already-redirected fd 1, so both the file name and the error land in all.txt. The rule: write > file 2>&1, file first.

Shorthand for both streams with &> and &>>

ls todo.txt missing.txt &> both.txt
ls notes.txt nope.txt &>> both.txt
cat both.txt
output
ls: cannot access 'missing.txt': No such file or directory
todo.txt
ls: cannot access 'nope.txt': No such file or directory
notes.txt

&> file is bash shorthand for > file 2>&1, and &>> file appends both streams. They are convenient in bash scripts, but they are not POSIX: under dash (/bin/sh on Debian) cmd &> file is parsed as cmd & followed by > file, which runs the command in the background and truncates the file. Use the long form in #!/bin/sh scripts.

Throwing output away

/dev/null accepts and discards anything written to it, and reads from it return end of file immediately. Redirecting to it is the standard way to silence a command when only its exit status matters.

Keep only the exit status

if ls missing.txt > /dev/null 2>&1; then
  echo "found"
else
  echo "not found, and no error message was printed"
fi
output
not found, and no error message was printed

Both streams are discarded, so the if sees only the exit status. For a quiet search, many commands have their own flag (grep -q), which is clearer and can stop early.

Input from a file: <

< connects a file to fd 0. The difference from passing the file name as an argument is visible in tools that print file names: when the data arrives on standard input, the command has no name to print.

Feed a file to standard input

wc -l todo.txt
wc -l < todo.txt
output
5 todo.txt
5

With an argument, wc opens the file itself and prints its name next to the count. With <, the shell opened it and wc just reads fd 0, so only the number appears. That bare number is handy in scripts: count=$(wc -l < todo.txt).

The classic accident: reading and writing the same file

Because the shell sets up redirections before the command starts, > file has already emptied the file by the time the command opens it for reading.

Sorting a file into itself empties it

sort todo.txt > todo.txt
wc -c todo.txt
output
0 todo.txt

The file is now 0 bytes. The shell truncated todo.txt to open it for output, then sort read an empty file. Write to a temporary file and rename it (sort todo.txt > tmp && mv tmp todo.txt), or use a tool's in-place option such as sort -o todo.txt todo.txt, which reads all input before opening the output.

Guarding against overwrites: noclobber

Refuse to overwrite existing files

set -o noclobber
echo "oops" > todo.txt
echo "exit status: $?"
echo "on purpose" >| todo.txt
cat todo.txt
output
/bin/bash: line 2: todo.txt: cannot overwrite existing file
exit status: 1
on purpose

With noclobber set, > refuses to truncate an existing regular file and the redirection fails, so the command never runs and the status is 1. >| overrides the setting for one redirection. Some people enable noclobber in their interactive ~/.bashrc to protect against typos.

Redirecting a whole block or a whole script

A redirection after { ...; } (or after a loop's done) applies to everything inside, and the file is opened only once. exec with only redirections and no command changes the descriptors of the current shell itself, for every command that follows.

Redirect a group, then the rest of the script

{ echo "report"; date +%Y; uname -s; } > report.txt
exec 3>> audit.log
echo "step one done" >&3
echo "step two done" >&3
exec 3>&-
cat report.txt audit.log
output
report
2026
Linux
step one done
step two done

The braces send three commands' output into one file. exec 3>> audit.log opens a new descriptor 3 for appending that stays open for the rest of the shell; >&3 writes to it without reopening the file each time, and 3>&- closes it. Scripts often start with exec > run.log 2>&1 to capture everything they print from that point on.

Swapping streams and piping only errors

A pipe connects only fd 1. To filter error messages, point fd 2 at the pipe and send the original fd 1 somewhere else. Remember the left-to-right rule: 2>&1 copies the pipe (fd 1 is already the pipe when redirections are applied), then > /dev/null moves fd 1 away.

Send only standard error through a pipe

ls todo.txt missing.txt nope.txt 2>&1 > /dev/null | tr a-z A-Z
explain
output
LS: CANNOT ACCESS 'MISSING.TXT': NO SUCH FILE OR DIRECTORY
LS: CANNOT ACCESS 'NOPE.TXT': NO SUCH FILE OR DIRECTORY

The listing of todo.txt went to /dev/null; the two error lines went through the pipe and were upper-cased by tr. To swap the two streams instead, use a spare descriptor: cmd 3>&1 1>&2 2>&3 3>&-.

Quick reference

Form Meaning
> f fd 1 to f, truncating
>> f fd 1 to f, appending
2> f fd 2 to f
> f 2>&1 both to f (POSIX)
&> f, &>> f both to f (bash only)
< f fd 0 from f
n>&m make fd n a copy of fd m
n>&- close fd n
>\| f overwrite even with noclobber

The full table with every operator is on the redirection cheat sheet. Here-documents (<<) are input redirections too; they have their own page, Here-documents and here-strings, and <(cmd) is covered in Process substitution.

Related