Bash Scripting Basics: Variables, Loops and Error Handling
A shell script automates a sequence of commands. Learn the syntax, the quoting rules, and the error handling that prevents surprises.
By Velloreth
The header
Every script should start with a shebang and strict mode.
#!/usr/bin/env bash
set -euo pipefail
set -e exits when a command fails.
set -u errors on an undefined variable, which catches typos.
set -o pipefail makes a pipeline fail if any command in it fails, not just the last one.
Without these, a script continues after a failure and produces confusing results. With them, it stops at the first problem.
Variables
name="world"
echo "Hello, $name"
# Command substitution
today=$(date +%F)
# Default value
port="${PORT:-3000}"
# Required value
: "${API_KEY:?API_KEY must be set}"
# Arithmetic
count=$((count + 1))
No spaces around =. name = "world" is a syntax error, because the shell parses it as a command.
Quote the expansion. "$name" not $name. An unquoted variable with spaces splits into multiple arguments, which is the most common bash bug.
file="my document.txt"
rm $file # tries to remove "my" and "document.txt"
rm "$file" # correct
Conditionals
if [[ -f "$file" ]]; then
echo "exists"
elif [[ -d "$file" ]]; then
echo "is a directory"
else
echo "missing"
fi
Use [[ ]] in bash. It handles quoting and supports pattern matching.
[[ "$name" == *.txt ]] # pattern match
[[ "$name" =~ ^[0-9]+$ ]] # regex
[[ -z "$var" ]] # empty
[[ -n "$var" ]] # non-empty
[[ "$a" -eq "$b" ]] # numeric equality
[[ "$a" -lt "$b" ]] # numeric less than
[[ "$a" == "$b" ]] # string equality
File tests:
| Test | True when |
|---|---|
-e | Exists |
-f | Regular file |
-d | Directory |
-r | Readable |
-w | Writable |
-x | Executable |
-s | Non-empty |
Loops
# Over a list
for item in a b c; do
echo "$item"
done
# Over files
for file in *.txt; do
echo "$file"
done
# C-style
for ((i = 0; i < 10; i++)); do
echo "$i"
done
# While
while read -r line; do
echo "$line"
done < file.txt
# Until
until [[ -f done.txt ]]; do
sleep 1
done
read -r prevents backslash interpretation, which is almost always what you want.
Reading a file line by line with while read is the correct pattern. for line in $(cat file) breaks on whitespace.
Functions
log() {
echo "[$(date +%T)] $*" >&2
}
die() {
echo "Error: $*" >&2
exit 1
}
log "starting"
[[ -f config.json ]] || die "config.json not found"
local for function-scoped variables.
process() {
local input="$1"
local result
result=$(echo "$input" | tr '[:lower:]' '[:upper:]')
echo "$result"
}
Return values. A function returns an exit code, not a value. To return a value, print it and capture it.
get_port() {
echo 3000
}
port=$(get_port)
Arguments
#!/usr/bin/env bash
set -euo pipefail
if [[ $# -lt 1 ]]; then
echo "Usage: $0 <file>" >&2
exit 1
fi
file="$1"
shift
$# is the argument count. $1, $2 are positional. "$@" is all arguments as separate words. $0 is the script name.
shift removes the first argument, which is useful for parsing flags.
while [[ $# -gt 0 ]]; do
case "$1" in
-v|--verbose) verbose=1; shift ;;
-o|--output) output="$2"; shift 2 ;;
*) die "Unknown option: $1" ;;
esac
done
Error handling
# Exit on error
set -e
# Check a specific command
if ! command -v jq &>/dev/null; then
die "jq is required"
fi
# Allow a command to fail
if ! grep -q "pattern" file.txt; then
echo "not found, continuing"
fi
# Cleanup on exit
cleanup() {
rm -f "$tmpfile"
}
trap cleanup EXIT
trap ... EXIT runs the cleanup whether the script succeeds or fails, which is the right place to remove temporary files.
command -v checks whether a command exists, which is more portable than which.
Debugging
bash -x script.sh # print each command
set -x # enable tracing in the script
set +x # disable
ShellCheck is a static analyser that catches most common mistakes.
shellcheck script.sh
It flags unquoted variables, useless cat, incorrect test syntax, and many other issues. Running it before committing a script is worth the habit.
Common patterns
Check for a required command:
for cmd in curl jq git; do
command -v "$cmd" &>/dev/null || die "$cmd is required"
done
Temporary directory:
tmpdir=$(mktemp -d)
trap 'rm -rf "$tmpdir"' EXIT
Read a config value:
value=$(grep -E '^key=' config.ini | cut -d= -f2-)
Retry a command:
retry() {
local n=0
until "$@"; do
n=$((n + 1))
[[ $n -ge 3 ]] && return 1
sleep 2
done
}
retry curl -fsS https://example.com
Arrays
Bash arrays hold a list of values.
files=("a.txt" "b.txt" "c.txt")
echo "${files[0]}" # first element
echo "${files[@]}" # all elements
echo "${#files[@]}" # length
echo "${files[-1]}" # last element
files+=("d.txt") # append
for f in "${files[@]}"; do
echo "$f"
done
Quote "${files[@]}". Unquoted, an element with a space splits into two.
Associative arrays (bash 4+):
declare -A config
config[host]="localhost"
config[port]="5432"
echo "${config[host]}"
for key in "${!config[@]}"; do
echo "$key = ${config[$key]}"
done
String manipulation
str="Hello, World"
echo "${#str}" # length
echo "${str,,}" # lowercase
echo "${str^^}" # uppercase
echo "${str:0:5}" # substring
echo "${str/World/There}" # replace first
echo "${str//l/L}" # replace all
echo "${str#*, }" # remove shortest prefix
echo "${str##*, }" # remove longest prefix
echo "${str%, *}" # remove shortest suffix
echo "${str%%, *}" # remove longest suffix
The prefix and suffix removals are the idiomatic way to strip a path or an extension.
path="/var/log/app.log"
echo "${path##*/}" # app.log
echo "${path%/*}" # /var/log
echo "${path%.log}" # /var/log/app
Reading input
# Prompt for input
read -rp "Enter your name: " name
echo "Hello, $name"
# Read a password without echoing
read -rsp "Password: " password
echo
# Read with a timeout
if read -rt 5 -p "Continue? " answer; then
echo "You said: $answer"
else
echo "Timed out"
fi
# Read into an array
read -ra parts <<< "one two three"
echo "${parts[1]}" # two
-r prevents backslash interpretation, which is almost always what you want. -s hides the input, for a password. -t sets a timeout.
Here documents
A here document passes a block of text as input.
cat <<EOF
Hello, $name
Today is $(date +%F)
EOF
Quote the delimiter to prevent expansion:
cat <<'EOF'
This $variable is not expanded
EOF
Write to a file:
cat > config.json <<EOF
{
"port": $port
}
EOF
A here string passes a single string as input.
grep "pattern" <<< "$variable"
Arithmetic and comparisons
# Arithmetic
result=$((5 + 3))
((count++))
((count += 5))
# Numeric comparison
if (( count > 10 )); then
echo "big"
fi
# Floating point needs bc or awk
result=$(echo "scale=2; 10 / 3" | bc)
result=$(awk "BEGIN {printf \"%.2f\", 10/3}")
Bash arithmetic is integer-only. For floating point, use bc or awk, which is a common surprise.
Common mistakes
An unquoted variable. rm $file with a space in the filename removes two files. Always quote: rm "$file".
No set -e. The script continues after a failure, and the error surfaces later in a confusing place.
[ ] instead of [[ ]]. [ ] is a command, so an unquoted variable with a space breaks it. Use [[ ]] in bash.
$* instead of $@. "$*" joins the arguments into one string, so a loop over them runs once. Use "$@".
A missing local. A variable in a function is global by default, which causes surprising interactions.
cd without checking. A failed cd leaves the script in the wrong directory, and the following commands operate on the wrong files.
cd /some/dir || exit 1
A pipeline that hides a failure. Without pipefail, only the last command’s exit code matters.
Comparing numbers as strings. [[ "10" > "9" ]] is a string comparison and is false. Use -gt or (( )).
A for loop over $(cat file). It splits on whitespace, so a line with a space becomes two iterations. Use while read.
Portability
#!/bin/sh versus #!/usr/bin/env bash. sh is a POSIX shell, which may be dash on Debian. Bash-specific syntax like [[ ]] and arrays does not work in sh.
Use #!/usr/bin/env bash when the script needs bash, and #!/bin/sh when it is genuinely POSIX.
env finds bash on the PATH, which is more portable than a hardcoded /bin/bash.
Check with ShellCheck using the right shell directive.
shellcheck -s bash script.sh
shellcheck -s sh script.sh
Avoid GNU-specific flags if the script must run on macOS, where the BSD versions of sed, grep, and date differ. sed -i needs an argument on macOS, and date -d does not exist.
Testing a script
A shell script is code, and it deserves the same care as any other code. The cheapest test is a dry run: add a --dry-run flag that prints the commands instead of running them, and use it before a destructive operation. The next step is to make the script’s behaviour depend on its arguments rather than on the environment, so a test can call it with known inputs. A script that reads a hardcoded path is hard to test; one that takes the path as an argument is easy. For anything beyond a few dozen lines, a test framework such as bats runs the script in a controlled environment and asserts on its output and exit code. The investment pays off the first time a change breaks something subtle, because the test catches it before the script runs against real data.
Scheduling a script
A script that should run on a schedule is usually driven by cron, and the environment cron provides is minimal. The PATH is short, the working directory is the home directory, and the shell is /bin/sh rather than bash. A script that works when you run it by hand can fail under cron for any of those reasons. The fix is to be explicit: use an absolute path to the interpreter, set the PATH at the top of the script, cd to the directory you expect, and redirect the output to a log file so you can see what happened. A cron job that produces no output and no log is impossible to debug, which is why the redirect is worth the line. Testing the exact command cron will run, in the same minimal environment, is the fastest way to find the difference.
Practical guidance
- Start with
set -euo pipefail. - Quote every variable expansion.
- Use
[[ ]]for conditionals. - Use
"$@"for arguments. - Use
localin functions. - Use
trap ... EXITfor cleanup. - Run ShellCheck.
- Use
bash -xto debug.
For the related guides, see linux-command-line-basics and cron-expressions-explained. For the broader context, environment-variables-explained and ci-cd-pipeline-basics.
Quotable summary
Start every script with set -euo pipefail, which stops on errors, undefined variables, and failed pipes, turning silent failures into loud ones. Always quote variable expansions, because an unquoted variable with spaces splits into multiple arguments, which is the most common bash bug. Use [[ ]] for conditionals in bash, which handles quoting and pattern matching better than [ ], and use ”$@” rather than ”$*” for arguments. Check exit codes with $? and use trap to clean up on exit, and run ShellCheck, which catches most of the common mistakes before you run the script.