opscape

Bash scripting guide · part 27 of 32

Brace expansion

Generate lists of words with {a,b,c} and sequences with {1..10} and {01..10..2}, why {1..$n} does not work, and how braces differ from globs.

Brace expansion is the first expansion bash performs on a command line, and the simplest: it generates words from a pattern, purely as text. It never looks at the file system, so it produces names whether or not files by those names exist. That is the main difference from globbing (*.txt), which only matches files that are already there.

There are two forms: a list {a,b,c} and a sequence {x..y} or {x..y..step}.

Lists

Expand a comma-separated list

echo file.{txt,csv,json}
echo {web,db}01
output
file.txt file.csv file.json
web01 db01

The text before and after the braces (the preamble and postscript) is attached to every alternative, giving one word per item. The braces themselves disappear.

The rename and backup shortcut

cp -v todo.txt{,.bak}
mv -v notes.txt{,.old}
ls todo.txt* notes.txt*
output
'todo.txt' -> 'todo.txt.bak'
renamed 'notes.txt' -> 'notes.txt.old'
notes.txt.old
todo.txt
todo.txt.bak

todo.txt{,.bak} expands to todo.txt todo.txt.bak, because the first alternative is empty. That is a copy to a backup in one short word. The mv form renames a file by adding a suffix without typing the name twice.

Sequences

{start..end} counts integers or single letters, in either direction. A third number sets the step. If either endpoint has leading zeros, bash pads every result to the same width, which keeps generated file names sorting correctly.

Number and letter ranges

echo {1..5}
echo {5..1}
echo {a..e}
echo {0..20..5}
echo {01..10..3}
output
1 2 3 4 5
5 4 3 2 1
a b c d e
0 5 10 15 20
01 04 07 10

Ranges count down when the start is larger, steps of 5 skip values, and {01..10..3} produced two-digit, zero-padded numbers: 01 04 07 10.

Several braces multiply

echo {a,b}{1,2}
mkdir -p proj/{src,docs}/{v1,v2}
find proj | sort
output
a1 a2 b1 b2
proj
proj/docs
proj/docs/v1
proj/docs/v2
proj/src
proj/src/v1
proj/src/v2

Adjacent braces produce every combination, the first one varying slowest. One mkdir -p created a 2 by 2 directory tree; this is the most common practical use of brace expansion.

Sequences in loops

for i in {1..10} is the idiomatic counting loop for a fixed range. Keep in mind that the whole list is generated up front, as words in memory, before the loop starts: {1..10000000} builds ten million words first, which takes noticeable time and memory. For large or open ranges, the arithmetic loop for ((i = 1; i <= n; i++)) costs nothing extra.

Create numbered files in a loop

for i in {1..3}; do
  echo "entry $i" > "log-$i.txt"
done
touch report-{01..12}.csv
ls log-* report-0*
output
log-1.txt
log-2.txt
log-3.txt
report-01.csv
report-02.csv
report-03.csv
report-04.csv
report-05.csv
report-06.csv
report-07.csv
report-08.csv
report-09.csv

The loop wrote three files. touch with a padded sequence created twelve CSV files in one command; the ls shows the first nine, whose names sort correctly thanks to the leading zero.

Braces are expanded before variables

Bash performs brace expansion before parameter expansion. When it looks at {1..$n}, the text $n is not a number, so the pattern is not a valid sequence and is left as it is. Only afterwards is $n replaced.

A variable in a range does not work

n=3
echo {1..$n}
seq 1 "$n" | paste -sd ' '
for ((i = 1; i <= n; i++)); do printf '%s ' "$i"; done; echo
output
{1..3}
1 2 3
1 2 3

{1..$n} came out literally as {1..3}. For a range whose end is in a variable, use seq or an arithmetic for loop. (eval echo {1..$n} also works, but eval on anything that could contain user input is a security hole.)

When braces are not expanded

A brace pattern needs either a comma or a valid .. range, and the braces must be unquoted. Anything else is left as it is. {} survives untouched, which is why find -exec cmd {} \; works without quoting.

Patterns that stay literal

echo {a} {} "{1..3}" '{x,y}' {a,}
echo "x"{1,2}"y"
output
{a} {} {1..3} {x,y} a
x1y x2y

A single item {a}, the empty {} and quoted braces print as typed. {a,} expands to a plus an empty word, and an empty unquoted word disappears from the command line, so only a remains. Quoting the parts around the braces is fine: "x"{1,2}"y" still expands.

Spaces end the expression

A brace expression is one word. An unquoted space inside it splits it into separate words before brace expansion ever runs, so {a, b} is not a list. Quote the space if an item needs one.

A space breaks the pattern, a quoted one does not

printf '[%s]\n' {a, b}
printf '[%s]\n' {"a b",c}
output
[{a,]
[b}]
[a b]
[c]

{a, b} became the two words {a, and b}, neither a valid pattern. In the second line the space is quoted, so the expression stays one word and expands to a b and c.

Braces and globs together

Because brace expansion runs first, *.{png,jpg} becomes *.png *.jpg, and each of those is then globbed separately. A glob that matches nothing stays literal (see Globbing), so a missing extension leaves a stray pattern on the command line.

Brace expansion feeding a glob

echo photos/*.{png,jpg}
explain
output
photos/IMG_2041.png photos/beach.png photos/forest.png photos/*.jpg

photos/*.png matched the three images. photos/*.jpg matched nothing and was passed on literally, which a command like rm would then complain about. Under shopt -s nullglob it would vanish instead.

Portability

Brace expansion is a bash, zsh and ksh feature. POSIX sh does not have it, and dash passes the braces through unchanged instead of failing, which makes the mistake easy to miss.

dash leaves braces alone

echo {1..3} file{,.bak}
explain
output
{1..3} file{,.bak}

No error, no expansion: the words are printed as typed. A #!/bin/sh script that relies on mkdir -p dir/{a,b} creates one directory literally named {a,b}. See POSIX sh vs bash.

Related