Bash scripting guide · part 1 of 32
The shebang line
What the kernel does with #!, why #!/usr/bin/env bash and #!/bin/bash both exist, and the ways a shebang line breaks.
The first line of most scripts starts with #!, read "shebang". It is not a shell feature.
When you run an executable file, the kernel looks at its first two bytes. A compiled program
starts with the ELF magic number and is loaded directly. A file that starts with #! is
handed to another program instead: the kernel reads the rest of the line as a path and runs
that program, passing it the script's path as an argument.
So for a file deploy.sh that begins #!/bin/bash, typing ./deploy.sh prod makes the kernel
run /bin/bash ./deploy.sh prod. Bash then reads the file, sees the first line as a comment
(it starts with #) and runs the rest. That is the whole trick, and it is why the same
mechanism works for Python, awk, or any program that treats # as a comment.
What the kernel passes to the interpreter
The clearest way to see the rewrite is to use a program that just prints its arguments as the "interpreter".
See the command line the kernel builds
#!/bin/echo
this line is never read
chmod +x show.sh
./show.sh first second
./show.sh first second
The kernel ran /bin/echo ./show.sh first second: the interpreter, then the path the script
was invoked by, then your arguments. echo never opens the file, so the second line is
ignored entirely.
A shebang line has exactly two parts: an absolute path to the interpreter and, optionally, one argument after it. On Linux, everything after the first run of spaces or tabs following the path, up to the end of the line, becomes that single argument, spaces included.
Everything after the path is one argument on Linux
#!/usr/bin/printf [%s]\n -x -y
chmod +x args.sh
./args.sh
[./args.sh] -x -y
printf received [%s]\n -x -y as its format string (one argument), then the script path.
The %s was filled with ./args.sh, and -x -y was printed literally on the next line as
part of the format. Had the kernel split the line, -x and -y would have been separate
arguments and printf would have formatted them too.
This rule matters as soon as you want to pass options. #!/bin/bash -eu works, because
-eu is one argument and bash accepts clustered options. But #!/bin/bash -e -u gives bash
the single argument -e -u, and #!/usr/bin/env bash -e asks env to find a program named
bash -e, with a space in its name.
#!/bin/bash or #!/usr/bin/env bash
#!/bin/bash hard-codes a location. It is fast and immune to PATH, but bash is not at
/bin/bash everywhere: NixOS has no /bin/bash at all, FreeBSD and OpenBSD install it under
/usr/local/bin, and on macOS /bin/bash is the old 3.2 release while a current bash from
Homebrew lives in /opt/homebrew/bin or /usr/local/bin.
#!/usr/bin/env bash runs env, which searches PATH for bash and runs the first one it
finds. /usr/bin/env exists at that path on practically every Unix-like system, so the line
travels well, and the user's preferred bash wins.
A script that finds bash through PATH
#!/usr/bin/env bash
echo "bash $BASH_VERSION, running as: $0"
ps -o args= -p $$
chmod +x where.sh
./where.sh
bash 5.2.37(1)-release, running as: ./where.sh bash ./where.sh
env located /usr/bin/bash (on Debian /bin is a symlink to /usr/bin) and replaced
itself with it, so the running process is just bash with the script path as its argument.
ps shows bash ./where.sh: env is not left behind as an extra process.
The cost of env is that PATH now decides which program runs your script. That is usually
what you want on a developer machine, and a liability for scripts run with elevated rights
or in a stranger's environment: anyone who controls PATH controls the interpreter.
PATH decides what env runs
#!/bin/sh
echo "not the bash you wanted (got: $*)"
#!/usr/bin/env bash
echo "hello from real bash"
chmod +x fakebin/bash hello.sh
./hello.sh
PATH="$PWD/fakebin:$PATH" ./hello.sh
hello from real bash not the bash you wanted (got: ./hello.sh)
With a directory containing a fake bash at the front of PATH, the same script runs the
impostor, which receives the script path as its argument. A #!/bin/bash script would not be
affected.
A reasonable rule: use #!/usr/bin/env bash for scripts that are shared across machines and
run by their own users, and an absolute path for system scripts, anything run by root from
cron or a service, and container images where you know exactly where bash is.
Passing options through env: env -S
Because the kernel delivers one argument, env needs help to receive both bash and an
option. GNU coreutils 8.30 and later, and the env of FreeBSD and macOS, have -S, which
splits its argument on spaces itself.
Options after env bash fail
#!/usr/bin/env bash -u
echo "value: $undefined"
chmod +x strict.sh
./strict.sh
env: ‘bash -u’: No such file or directory env: use -[v]S to pass options in shebang lines
env was asked to run a program literally named bash -u and found none, so it exits
with 127, the "command not found" status. GNU env adds a hint pointing at -S.
The same shebang with env -S
#!/usr/bin/env -S bash -u
echo "value: $undefined"
echo "not reached"
chmod +x strict.sh
./strict.sh
./strict.sh: line 2: undefined: unbound variable
-S turned bash -u into two words, so bash started with -u and stopped at the first
reference to an unset variable. A simpler, fully portable alternative is to keep a plain
shebang and put set -u (or set -euo pipefail) on the second line.
#!/bin/sh is not bash
#!/bin/sh asks for the system's POSIX shell. On Debian and Ubuntu that is dash, a small
fast shell without bash extensions; on macOS it is bash in POSIX mode, and on Alpine it is
BusyBox ash. A script that uses [[ ]], arrays, source or ${var//x/y} under a #!/bin/sh
line may work on one machine and break on the next. The fixture's scripts/greet.sh uses
#!/bin/sh correctly: it only uses POSIX features.
A bash-only test under #!/bin/sh
#!/bin/sh
name=world
if [[ $name == w* ]]; then
echo "starts with w"
fi
echo "BASH_VERSION is '${BASH_VERSION}'"
chmod +x check.sh
./check.sh
readlink -f /bin/sh
./check.sh: 3: [[: not found BASH_VERSION is '' /usr/bin/dash
dash does not know [[, so it tries to run a command named [[, which does not exist, and
the if is simply false. The script keeps going and exits 0, which is why these bugs
survive testing. BASH_VERSION is empty because no bash is involved, and readlink
shows /bin/sh resolves to dash.
Either change the shebang to bash or write POSIX code; the POSIX sh vs bash page
lists the usual substitutions ([ ] for [[ ]], case for pattern tests, . for source).
No shebang at all
Without #!, the kernel does not recognize the file and the execve system call fails with
ENOEXEC, "Exec format error". What happens next depends on who asked. Shells catch the
error and run the file as a shell script themselves: bash runs it in a bash subshell, dash
in dash. The C library's execvp, used by env, xargs and many other programs, falls back
to /bin/sh. A program that calls execve directly just gets the error.
One file, three different outcomes
echo "BASH_VERSION='${BASH_VERSION}'"
chmod +x plain.sh
./plain.sh
env ./plain.sh
python3 -c 'import subprocess; subprocess.run(["./plain.sh"])' 2>&1 | tail -n 1
BASH_VERSION='5.2.37(1)-release' BASH_VERSION='' OSError: [Errno 8] Exec format error: './plain.sh'
Started from bash, the script ran in bash and BASH_VERSION is set. Through env
(execvp), it was run by /bin/sh, which is dash, so the variable is empty. Python's
subprocess calls execve without a fallback and reports Exec format error. Always
write the shebang; then all three agree.
Windows line endings: bad interpreter
A script saved with CRLF line endings has an invisible carriage return at the end of the
shebang line. The kernel takes it as part of the interpreter's path and looks for a file
literally named /bin/bash\r.
A CRLF script fails before it starts
printf '#!/bin/bash\r\necho hello\r\n' > win.sh
chmod +x win.sh
./win.sh; echo "exit status: $?"
head -n 1 win.sh | od -c
/bin/bash: line 3: ./win.sh: cannot execute: required file not found exit status: 127 0000000 # ! / b i n / b a s h \r \n 0000015
Bash 5.2 says cannot execute: required file not found, status 127, even though both
the script and /bin/bash exist, which is the giveaway. Older bash releases, and most
answers you will find online, show the same failure as /bin/bash^M: bad interpreter: No
such file or directory, where ^M is the carriage return. od -c makes it visible as
\r right before the newline.
Strip the carriage returns
printf '#!/bin/bash\r\necho hello\r\n' > win.sh
chmod +x win.sh
sed -i 's/\r$//' win.sh
./win.sh
od -c win.sh | head -n 2
hello 0000000 # ! / b i n / b a s h \n e c h o 0000020 h e l l o \n
sed -i 's/\r$//' removes one carriage return at the end of each line (dos2unix does the
same if installed). The od -c dump confirms only \n remains. Configure your editor or
Git (core.autocrlf, .gitattributes) so they do not come back.
Other things that silently break a shebang: anything before the #!, including a blank line
or a UTF-8 byte order mark that some Windows editors add; and a missing execute bit, which
gives Permission denied (exit 126) no matter what the first line says. Linux also limits
the shebang line's length (256 bytes since kernel 5.1, 128 before), which deep
virtualenv paths occasionally hit.
When the shebang is ignored
The shebang only matters when the kernel executes the file. Running bash script.sh names
the interpreter explicitly, so the first line is just a comment. Sourcing with source or .
runs the file's lines inside the current shell, again without looking at #!.
bash script.sh and source both skip the shebang
#!/usr/bin/python3
echo "this is bash code, run by ${BASH_VERSION:+bash}"
bash py.sh
source py.sh
chmod +x py.sh
./py.sh
this is bash code, run by bash
this is bash code, run by bash
File "/home/dev/demo/./py.sh", line 2
echo "this is bash code, run by ${BASH_VERSION:+bash}"
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
SyntaxError: invalid syntax The first two runs ignore #!/usr/bin/python3 and execute the echo as bash. Only ./py.sh
consults the shebang, so Python gets the file and rejects the shell syntax with a
SyntaxError. Make the shebang match what the file really contains.
Checklist
- First line, first byte:
#!followed by an absolute path. No blank line or BOM before it. #!/usr/bin/env bashfor portable user scripts;#!/bin/bash(or wherever bash is) for system scripts wherePATHshould not matter.- Need options? Put
set -euo pipefailon line two instead of in the shebang, or useenv -S. - Use
#!/bin/shonly for code you have actually run under dash. - Unix line endings, and
chmod +x. How to start the script is in Running scripts.
Related
envPrint the environment, or run a command with a modified environment.fileGuess what kind of data a file holds by looking at its contents, not its name.odDump a file's bytes as octal, hex, decimal or characters, to see exactly what it contains.- Running scriptsThe four ways to start a script (./script, bash script, PATH, source), what each one changes, and the errors you get when one goes wrong.
- POSIX sh vs bashThe bashisms that break when a script runs under dash or another POSIX sh, the exact errors dash prints, and the portable replacement for each.
- Strict mode and its caveatsWhat set -euo pipefail actually does, the cases where -e silently does nothing, the traps in -u and pipefail, and a balanced way to use it.