Here-documents and here-strings: bash scripting guide

A here-document is an input redirection whose content is written inline, in the script, instead of coming from a file. The shell reads the lines that follow the command up to a line containing only the delimiter word, and connects that text to the command's standard input.

command <<WORD
line one
line two
WORD

WORD can be anything; EOF, END and SQL are common choices. The command is any command that reads standard input: cat, mysql, ssh host bash, python3 -, a while read loop.

Expansion happens by default

A here-document with variables and command substitution

$ name=demo
> cat <<EOF
> Project: $name
> Tasks open: $(wc -l < todo.txt)
> Home is $HOME
> EOF

output

Project: demo
Tasks open: 5
Home is /home/dev

With an unquoted delimiter, the body is treated like a double-quoted string: $name, $(...) and $HOME are expanded before cat sees the text. Word splitting and globbing do not happen, so spacing inside the lines is kept exactly.

Quoting the delimiter turns expansion off

Put any quotes on the delimiter ('EOF', "EOF" or even \EOF) and the body is passed through literally. This is the form you want when the text is itself code: a script you are generating, a SQL query with $1 placeholders, an awk program.

Generate a script without expanding anything

$ cat > hello.sh <<'EOF'
> #!/bin/bash
> echo "Hello from $(hostname), user $USER"
> EOF
> cat hello.sh
> echo "--- running it:"
> bash hello.sh

output

#!/bin/bash
echo "Hello from $(hostname), user $USER"
--- running it:
Hello from demo, user dev

The file contains $(hostname) and $USER exactly as typed, because the quoted 'EOF' disabled expansion. They are expanded later, when hello.sh runs. With an unquoted EOF they would have been expanded once, while writing the file, and frozen into it.

Expand some things but not others

$ dir=/srv/app
> cat <<EOF
> cd $dir
> echo "started as \$USER at \$(date +%H:%M)"
> EOF

output

cd /srv/app
echo "started as $USER at $(date +%H:%M)"

In an unquoted here-document, a backslash protects a single $, backtick or backslash. $dir was filled in now; \$USER and \$(date ...) came out literally, ready to be evaluated by whatever runs this text later.

The delimiter line must match exactly

The closing word has to be alone on its line, at the very start, with no trailing spaces. If it is indented with spaces, or followed by a stray character, bash keeps reading to the end of the script and warns you.

An indented delimiter is not recognized

$ if true; then
>   cat <<EOF
>   inside the if
>   EOF
> fi
> echo "this line was swallowed too"

outputexit 2

/bin/bash: line 6: warning: here-document at line 2 delimited by end-of-file (wanted `EOF')
/bin/bash: -c: line 7: syntax error: unexpected end of file

EOF with leading spaces is not the delimiter, so the here-document runs on to the end of the input, swallowing fi and the final echo. Bash warns, then reports a syntax error because the if was never closed, so nothing in the block runs at all. In a longer script this shows up as a confusing error at the last line of the file.

Indenting with <<-

<<- strips leading tab characters (not spaces) from every body line and from the delimiter line. It exists so that here-documents inside functions and loops can follow the code's indentation. It only works if the indentation really is tabs, which many editors silently convert to spaces; cat -A shows tabs as ^I.

Strip tab indentation with <<-

usage.sh

usage() {
	cat <<-EOF
		usage: deploy [-n] TARGET
		  -n  dry run
	EOF
}
usage

$ cat -A usage.sh | head -n 4
> echo "---"
> bash usage.sh

output

usage() {$
^Icat <<-EOF$
^I^Iusage: deploy [-n] TARGET$
^I^I  -n  dry run$
---
usage: deploy [-n] TARGET
  -n  dry run

cat -A shows the indentation is tabs (^I) and marks line ends with $. Running the script, every leading tab is removed, so the usage text starts at column one. The two spaces before -n survive: <<- removes only tabs, and only at the start of the line.

Here-documents into other programs

Anything that reads a script from standard input can take a here-document. Quote the delimiter so the local shell does not expand variables meant for the other side.

Run a block of commands on a remote host

$ ssh server bash <<'EOF'
> echo "remote host: $(hostname)"
> cd /tmp && pwd
> EOF

output

remote host: demo
/tmp

The body travels over the SSH connection as the standard input of bash on the remote host, which runs it line by line. Because of 'EOF', $(hostname) was evaluated there, not locally. (In these examples the "remote" host is the sandbox itself, so the name matches.)

Feed a loop from a here-document

$ while read -r host port; do
>   echo "checking $host on port $port"
> done <<EOF
> web01 443
> db01 5432
> EOF

output

checking web01 on port 443
checking db01 on port 5432

The redirection goes after done, so the whole loop reads from it. Unlike printf ... | while read, the loop runs in the current shell, so variables set inside it are still set afterwards.

Here-strings: <<<

Bash (and zsh and ksh) also have the here-string, <<< word: the word is expanded, a newline is appended, and the result becomes standard input. It replaces the common echo "$var" | cmd, without a pipe or subshell.

Pass one string as standard input

$ line="web01 10.0.0.11 up"
> read -r name ip state <<< "$line"
> echo "name=$name ip=$ip state=$state"
> bc <<< "2^10"
> tr a-z A-Z <<< "shout this"

output

name=web01 ip=10.0.0.11 state=up
1024
SHOUT THIS

read splits the line into three variables in the current shell (with echo ... | read they would vanish with the pipeline's subshell). bc and tr read the strings as if they had come from a file.

A here-string adds a trailing newline

$ printf %s abc | wc -c
> wc -c <<< abc
> wc -c <<< ""

output

3
4
1

Three bytes from printf %s, four from the here-string, which appended a newline, and even the empty string produces one byte. That matters when the input is hashed or encoded: sha256sum <<< "$secret" hashes the secret plus \n; use printf %s "$secret" | sha256sum instead.

Here-document into a variable

To keep a multi-line block in a variable, wrap the here-document in command substitution. read -r -d '' var <<'EOF' also works, but read returns status 1 when it hits end of input before the (empty) delimiter, which kills a script running under set -e.

Store a block of text in a variable

$ query=$(cat <<'EOF'
> SELECT name, email
> FROM users
> WHERE dept = 'sales';
> EOF
> )
> printf '%s\n' "$query"
> read -r -d '' other <<'EOF'
> two
> lines
> EOF
> echo "read returned $?"

output

SELECT name, email
FROM users
WHERE dept = 'sales';
read returned 1

Command substitution captures the here-document's text and strips the trailing newline. The read -d '' form also filled other, but returned 1, which is why set -e scripts either use the $(cat <<...) form or write read -r -d '' other <<'EOF' || true.

Here-documents are POSIX and work in sh; <<- is POSIX too. Here-strings are not: under dash <<< is a syntax error (see POSIX sh vs bash). For the other input redirections see Redirection.

Expansion happens by default

Quoting the delimiter turns expansion off

The delimiter line must match exactly

Indenting with <<-

Here-documents into other programs

Here-strings: <<<

Here-document into a variable

Related