opscape

Bash scripting guide · part 1 of 32

The shebang line

What the kernel does with #!, why #!/usr/bin/env bash and #!/bin/bash both exist, and the ways a shebang line breaks.

The first line of most scripts starts with #!, read "shebang". It is not a shell feature. When you run an executable file, the kernel looks at its first two bytes. A compiled program starts with the ELF magic number and is loaded directly. A file that starts with #! is handed to another program instead: the kernel reads the rest of the line as a path and runs that program, passing it the script's path as an argument.

So for a file deploy.sh that begins #!/bin/bash, typing ./deploy.sh prod makes the kernel run /bin/bash ./deploy.sh prod. Bash then reads the file, sees the first line as a comment (it starts with #) and runs the rest. That is the whole trick, and it is why the same mechanism works for Python, awk, or any program that treats # as a comment.

What the kernel passes to the interpreter

The clearest way to see the rewrite is to use a program that just prints its arguments as the "interpreter".

See the command line the kernel builds

show.sh
#!/bin/echo
this line is never read
chmod +x show.sh
./show.sh first second
output
./show.sh first second

The kernel ran /bin/echo ./show.sh first second: the interpreter, then the path the script was invoked by, then your arguments. echo never opens the file, so the second line is ignored entirely.

A shebang line has exactly two parts: an absolute path to the interpreter and, optionally, one argument after it. On Linux, everything after the first run of spaces or tabs following the path, up to the end of the line, becomes that single argument, spaces included.

Everything after the path is one argument on Linux

args.sh
#!/usr/bin/printf [%s]\n -x -y
chmod +x args.sh
./args.sh
output
[./args.sh]
 -x -y

printf received [%s]\n -x -y as its format string (one argument), then the script path. The %s was filled with ./args.sh, and -x -y was printed literally on the next line as part of the format. Had the kernel split the line, -x and -y would have been separate arguments and printf would have formatted them too.

This rule matters as soon as you want to pass options. #!/bin/bash -eu works, because -eu is one argument and bash accepts clustered options. But #!/bin/bash -e -u gives bash the single argument -e -u, and #!/usr/bin/env bash -e asks env to find a program named bash -e, with a space in its name.

#!/bin/bash or #!/usr/bin/env bash

#!/bin/bash hard-codes a location. It is fast and immune to PATH, but bash is not at /bin/bash everywhere: NixOS has no /bin/bash at all, FreeBSD and OpenBSD install it under /usr/local/bin, and on macOS /bin/bash is the old 3.2 release while a current bash from Homebrew lives in /opt/homebrew/bin or /usr/local/bin.

#!/usr/bin/env bash runs env, which searches PATH for bash and runs the first one it finds. /usr/bin/env exists at that path on practically every Unix-like system, so the line travels well, and the user's preferred bash wins.

A script that finds bash through PATH

where.sh
#!/usr/bin/env bash
echo "bash $BASH_VERSION, running as: $0"
ps -o args= -p $$
chmod +x where.sh
./where.sh
output
bash 5.2.37(1)-release, running as: ./where.sh
bash ./where.sh

env located /usr/bin/bash (on Debian /bin is a symlink to /usr/bin) and replaced itself with it, so the running process is just bash with the script path as its argument. ps shows bash ./where.sh: env is not left behind as an extra process.

The cost of env is that PATH now decides which program runs your script. That is usually what you want on a developer machine, and a liability for scripts run with elevated rights or in a stranger's environment: anyone who controls PATH controls the interpreter.

PATH decides what env runs

fakebin/bash
#!/bin/sh
echo "not the bash you wanted (got: $*)"
hello.sh
#!/usr/bin/env bash
echo "hello from real bash"
chmod +x fakebin/bash hello.sh
./hello.sh
PATH="$PWD/fakebin:$PATH" ./hello.sh
output
hello from real bash
not the bash you wanted (got: ./hello.sh)

With a directory containing a fake bash at the front of PATH, the same script runs the impostor, which receives the script path as its argument. A #!/bin/bash script would not be affected.

A reasonable rule: use #!/usr/bin/env bash for scripts that are shared across machines and run by their own users, and an absolute path for system scripts, anything run by root from cron or a service, and container images where you know exactly where bash is.

Passing options through env: env -S

Because the kernel delivers one argument, env needs help to receive both bash and an option. GNU coreutils 8.30 and later, and the env of FreeBSD and macOS, have -S, which splits its argument on spaces itself.

Options after env bash fail

strict.sh
#!/usr/bin/env bash -u
echo "value: $undefined"
chmod +x strict.sh
./strict.sh
outputexit 127
env: ‘bash -u’: No such file or directory
env: use -[v]S to pass options in shebang lines

env was asked to run a program literally named bash -u and found none, so it exits with 127, the "command not found" status. GNU env adds a hint pointing at -S.

The same shebang with env -S

strict.sh
#!/usr/bin/env -S bash -u
echo "value: $undefined"
echo "not reached"
chmod +x strict.sh
./strict.sh
outputexit 1
./strict.sh: line 2: undefined: unbound variable

-S turned bash -u into two words, so bash started with -u and stopped at the first reference to an unset variable. A simpler, fully portable alternative is to keep a plain shebang and put set -u (or set -euo pipefail) on the second line.

#!/bin/sh is not bash

#!/bin/sh asks for the system's POSIX shell. On Debian and Ubuntu that is dash, a small fast shell without bash extensions; on macOS it is bash in POSIX mode, and on Alpine it is BusyBox ash. A script that uses [[ ]], arrays, source or ${var//x/y} under a #!/bin/sh line may work on one machine and break on the next. The fixture's scripts/greet.sh uses #!/bin/sh correctly: it only uses POSIX features.

A bash-only test under #!/bin/sh

check.sh
#!/bin/sh
name=world
if [[ $name == w* ]]; then
  echo "starts with w"
fi
echo "BASH_VERSION is '${BASH_VERSION}'"
chmod +x check.sh
./check.sh
readlink -f /bin/sh
output
./check.sh: 3: [[: not found
BASH_VERSION is ''
/usr/bin/dash

dash does not know [[, so it tries to run a command named [[, which does not exist, and the if is simply false. The script keeps going and exits 0, which is why these bugs survive testing. BASH_VERSION is empty because no bash is involved, and readlink shows /bin/sh resolves to dash.

Either change the shebang to bash or write POSIX code; the POSIX sh vs bash page lists the usual substitutions ([ ] for [[ ]], case for pattern tests, . for source).

No shebang at all

Without #!, the kernel does not recognize the file and the execve system call fails with ENOEXEC, "Exec format error". What happens next depends on who asked. Shells catch the error and run the file as a shell script themselves: bash runs it in a bash subshell, dash in dash. The C library's execvp, used by env, xargs and many other programs, falls back to /bin/sh. A program that calls execve directly just gets the error.

One file, three different outcomes

plain.sh
echo "BASH_VERSION='${BASH_VERSION}'"
chmod +x plain.sh
./plain.sh
env ./plain.sh
python3 -c 'import subprocess; subprocess.run(["./plain.sh"])' 2>&1 | tail -n 1
output
BASH_VERSION='5.2.37(1)-release'
BASH_VERSION=''
OSError: [Errno 8] Exec format error: './plain.sh'

Started from bash, the script ran in bash and BASH_VERSION is set. Through env (execvp), it was run by /bin/sh, which is dash, so the variable is empty. Python's subprocess calls execve without a fallback and reports Exec format error. Always write the shebang; then all three agree.

Windows line endings: bad interpreter

A script saved with CRLF line endings has an invisible carriage return at the end of the shebang line. The kernel takes it as part of the interpreter's path and looks for a file literally named /bin/bash\r.

A CRLF script fails before it starts

printf '#!/bin/bash\r\necho hello\r\n' > win.sh
chmod +x win.sh
./win.sh; echo "exit status: $?"
head -n 1 win.sh | od -c
output
/bin/bash: line 3: ./win.sh: cannot execute: required file not found
exit status: 127
0000000   #   !   /   b   i   n   /   b   a   s   h  \r  \n
0000015

Bash 5.2 says cannot execute: required file not found, status 127, even though both the script and /bin/bash exist, which is the giveaway. Older bash releases, and most answers you will find online, show the same failure as /bin/bash^M: bad interpreter: No such file or directory, where ^M is the carriage return. od -c makes it visible as \r right before the newline.

Strip the carriage returns

printf '#!/bin/bash\r\necho hello\r\n' > win.sh
chmod +x win.sh
sed -i 's/\r$//' win.sh
./win.sh
od -c win.sh | head -n 2
output
hello
0000000   #   !   /   b   i   n   /   b   a   s   h  \n   e   c   h   o
0000020       h   e   l   l   o  \n

sed -i 's/\r$//' removes one carriage return at the end of each line (dos2unix does the same if installed). The od -c dump confirms only \n remains. Configure your editor or Git (core.autocrlf, .gitattributes) so they do not come back.

Other things that silently break a shebang: anything before the #!, including a blank line or a UTF-8 byte order mark that some Windows editors add; and a missing execute bit, which gives Permission denied (exit 126) no matter what the first line says. Linux also limits the shebang line's length (256 bytes since kernel 5.1, 128 before), which deep virtualenv paths occasionally hit.

When the shebang is ignored

The shebang only matters when the kernel executes the file. Running bash script.sh names the interpreter explicitly, so the first line is just a comment. Sourcing with source or . runs the file's lines inside the current shell, again without looking at #!.

bash script.sh and source both skip the shebang

py.sh
#!/usr/bin/python3
echo "this is bash code, run by ${BASH_VERSION:+bash}"
bash py.sh
source py.sh
chmod +x py.sh
./py.sh
outputexit 1
this is bash code, run by bash
this is bash code, run by bash
  File "/home/dev/demo/./py.sh", line 2
    echo "this is bash code, run by ${BASH_VERSION:+bash}"
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
SyntaxError: invalid syntax

The first two runs ignore #!/usr/bin/python3 and execute the echo as bash. Only ./py.sh consults the shebang, so Python gets the file and rejects the shell syntax with a SyntaxError. Make the shebang match what the file really contains.

Checklist

  • First line, first byte: #! followed by an absolute path. No blank line or BOM before it.
  • #!/usr/bin/env bash for portable user scripts; #!/bin/bash (or wherever bash is) for system scripts where PATH should not matter.
  • Need options? Put set -euo pipefail on line two instead of in the shebang, or use env -S.
  • Use #!/bin/sh only for code you have actually run under dash.
  • Unix line endings, and chmod +x. How to start the script is in Running scripts.

Related