Automating Boring Work with Shell Scripts
How to write shell scripts that survive real use — strict mode, quoting, dry runs, ShellCheck, and knowing when a script should have been a program.
By Velloreth
The case for shell scripting is embarrassingly simple: some things you do repeatedly are faster to write down than to remember. Renaming a batch of files, cleaning merged branches, scaffolding a new project the same way every time — none of it is hard, all of it eats attention.
The bar for automating is low on purpose. If you have done it three times by hand and expect to do it again, write it down. The first two repetitions taught you what the task actually involves; the third is where the script pays.
This is about the scripts that survive contact with reality — the ones you still trust a year later.
What is worth scripting
| Automate | Leave alone |
|---|---|
| Repetitive file operations with a pattern | One-off commands you will never repeat |
| Multi-step sequences you run in order from memory | Trivial commands shorter than the script’s usage text |
| Anything you have done three or more times | Logic with deep branching, JSON, or heavy data work |
| Checks you want to run the same way every time | Anything whose failure mode is “deletes data” without a dry run |
| Scheduled jobs (backups, reports, cleanup) | Interactions that need your judgment each time |
The second column matters. A script is a decision you make once and execute many times — which is exactly wrong for decisions that should be fresh each time.
The shape of a script that lasts
Five habits separate scripts you trust from scripts you re-read in a panic.
1. Strict mode and a real shebang
#!/usr/bin/env bash
set -euo pipefail
set -e exits on error, -u treats unset variables as errors instead of empty strings, and -o pipefail makes a pipeline fail when any stage fails — without it, grep | head hides the exit status of grep.
Use #!/usr/bin/env bash rather than #!/bin/bash: it finds bash on PATH, which matters on macOS and any machine with a non-standard install location.
2. Quote everything
# broken with spaces in the filename
rm $1
# correct
rm -- "$1"
Unquoted expansions undergo word splitting and globbing. A file named my file.txt becomes two arguments; a filename containing * becomes a directory listing. ShellCheck flags this as SC2086, and it is the single most common way scripts damage things.
The -- before arguments is cheap insurance against filenames starting with -.
3. Fail loudly, exit with meaning
die() { echo "error: $*" >&2; exit 1; }
[[ -d "$dir" ]] || die "not a directory: $dir"
command -v git >/dev/null || die "git is required"
Exit codes are the interface other tools use. 0 means success, non-zero means failure, and anything from 1–125 is yours to define. A script that exits 0 after failing is worse than no script — it teaches automation to trust it.
4. Ask before mutating
Every destructive script gets a dry-run mode:
dry_run=false
[[ "${1:-}" == "--dry-run" ]] && dry_run=true
run() {
if $dry_run; then echo "would run: $*"; else "$@"; fi
}
run rm -rf -- "$stale_dir"
Five lines, and suddenly you can point the script at the wrong directory and find out without consequences.
5. Help text
usage() {
cat >&2 <<EOF
Usage: $(basename "$0") [--dry-run] <directory>
Removes build artifacts older than 7 days.
EOF
exit 64
}
Future you counts as a user. usage also gives you somewhere to send bad arguments instead of failing halfway through.
A complete example
A script that cleans stale local git branches, with everything above applied:
#!/usr/bin/env bash
set -euo pipefail
usage() {
cat >&2 <<EOF
Usage: $(basename "$0") [--dry-run] [--merged]
Deletes local branches already merged into main.
EOF
exit 64
}
die() { echo "error: $*" >&2; exit 1; }
dry_run=false
merged_only=true
for arg in "$@"; do
case "$arg" in
--dry-run) dry_run=true ;;
--all) merged_only=false ;;
-h|--help) usage ;;
*) usage ;;
esac
done
command -v git >/dev/null || die "git is required"
git rev-parse --is-inside-work-tree >/dev/null 2>&1 || die "not a git repository"
current=$(git branch --show-current)
[[ -n "$current" ]] || die "detached HEAD; check out a branch first"
git fetch --prune --quiet
if $merged_only; then
mapfile -t branches < <(git branch --merged main |
sed "s/^[* ] *//" |
grep -v -e "^${current}$" -e '^main$' -e '^master$' || true)
else
mapfile -t branches < <(git branch |
sed "s/^[* ] *//" |
grep -v -e "^${current}$" -e '^main$' -e '^master$' || true)
fi
if ((${#branches[@]} == 0)); then
echo "nothing to clean"
exit 0
fi
for branch in "${branches[@]}"; do
if $dry_run; then
echo "would delete: $branch"
else
git branch -d "$branch" || git branch -D "$branch"
fi
done
Notice what it does before touching anything: checks that the required tool exists, checks that it is inside a repository, refuses to run on a detached HEAD, fetches prunes, excludes the current branch and the defaults, and reports nothing when there is nothing to do.
That is what “trust” looks like in shell. Not cleverness — checks in the right order.
Parsing arguments, three levels
The example above parses flags with a for/case loop, which is honest and easy to read. As flags multiply, move up a level.
Level 1 — a case loop (what the example uses): fine for two or three flags, and the only option if you want long names like --dry-run.
Level 2 — getopts, built into bash, handles short flags, clustering, and values:
while getopts ":ndh" opt; do
case "$opt" in
n) dry_run=true ;;
d) merged_only=false ;;
h) usage ;;
\?) echo "unknown option: -$OPTARG" >&2; usage ;;
:) echo "option -$OPTARG requires a value" >&2; usage ;;
esac
done
shift $((OPTIND - 1))
dir="${1:-.}"
The leading : in the optstring suppresses getopts’ own error messages so you can print your own, and the : case catches options given without their required value. What getopts does not do is long options — that stays a manual loop.
Level 3 — subcommands, when one script grows several jobs:
case "${1:-}" in
clean) shift; cmd_clean "$@" ;;
add) shift; cmd_add "$@" ;;
-h|--help|'') usage ;;
*) echo "unknown command: $1" >&2; usage ;;
esac
That is how tools like git and docker behave, and it is worth adopting the moment you find yourself writing --do-the-other-thing to distinguish two modes.
Portability traps you will hit
If your scripts ever run on macOS as well as Linux, the userland differs in ways that fail at the worst moment:
- BSD versus GNU flags.
sed -iwants a suffix argument on macOS (sed -i '' ...) and refuses one on GNU. BSDdatehas no-d "yesterday"; it usesdate -v-1d. BSDstatuses-f, GNU uses-c. - Avoid in-place edits entirely and sidestep the whole class:
sed 's/a/b/' file > "$file.tmp" && mv "$file.tmp" "$file". readlink -fis missing on older macOS. Either install coreutils (greadlink) or resolve paths manually.- bash itself. Stock macOS still ships bash 3.2, so bash-4-only features —
mapfile, associative arrays,${var,,}— break despite your shebang. Check with[[ ${BASH_VERSINFO[0]} -ge 4 ]], or write the portable subset when macOS is a target. - Set
LC_ALL=Cat the top of scripts that sort or parse text. It makes ordering and byte behavior deterministic regardless of the machine’s locale.
The pragmatic rule: write for the machines you actually run on, and if that set includes a stock macOS box, test there — not in your Linux container.
Testing before shipping
ShellCheck is not optional. It catches quoting errors, unquoted expansions, useless cat, deprecated syntax, and a hundred other things:
shellcheck my-script.sh
bash -n my-script.sh # syntax check only, instant
Install it from your package manager (apt install shellcheck, brew install shellcheck) and add it to pre-commit hooks. Roughly every shell bug I have personally shipped was one ShellCheck had already flagged.
Test against a scratch directory, never your real data:
mkdir -p /tmp/script-test && cd /tmp/script-test
git init sample && cd sample
git commit --allow-empty -m init
../my-script.sh --dry-run
Make it idempotent where you can. Creating a directory should use mkdir -p; installing a symlink should overwrite with ln -sf. Then running the script twice is safe, which makes it safe to run when you are unsure whether you already ran it.
For anything non-trivial, bats-core gives you a real test framework for shell:
@test "refuses to run outside git repos" {
run ./clean-branches.sh
[[ "$status" -eq 1 ]]
[[ "$output" == *"not a git repository"* ]]
}
You do not need this for a twenty-line helper. You do want it the day the script starts doing something you would not run manually without thinking.
Where scripts live
Keep them in your dotfiles repository and symlink them onto PATH:
~/dotfiles/home/.local/bin/
├── clean-branches
├── new-project
├── backup-notes
└── gif-record
# in .bashrc / .zshrc
export PATH="$HOME/.local/bin:$PATH"
Version control buys you history, sync across machines, and a place to look when a script does something surprising six months later. A scripts directory that is not in git is a scripts directory you will eventually lose.
Name scripts for what they do (clean-branches, not stuff2.sh), keep them executable (chmod +x), and put a two-line description in the header comment — the same discipline you would apply to any function someone else will read.
Scheduling the boring stuff
Once a script is trustworthy, the next step is not running it — it is not having to remember it.
Cron remains the simple answer:
0 3 * * * /home/me/.local/bin/backup-notes >> /home/me/.local/state/backup.log 2>&1
On systemd machines, a timer unit adds the parts cron lacks — Persistent=true runs missed jobs after downtime, and systemctl list-timers shows you what is queued and when it fires next.
Two rules for scheduled jobs: log somewhere you will actually look, and make failure loud (an email, a notification, a non-zero exit your monitoring sees). A job that fails silently for three months has cost you more than the automation saved.
Knowing when to stop
Shell is for orchestration: starting things, checking conditions, moving files, calling other tools. It gets uncomfortable past a page of logic — nested case statements, string parsing that should be JSON parsing, error handling that fights you at every turn.
The migration signals:
- You are writing functions to emulate data structures.
- The script parses structured data with
grepandcut. - Debugging means adding
echoin six places because there is no stack trace. - You hesitate before changing it.
At that point, rewrite it in a language you can test properly — Python, Node, whatever you already know — and leave the shell script as the thin wrapper that calls it. Nobody wins points for writing the complex thing in shell.
The goal was never shell mastery. It was to never manually do for the fourth time something a machine can do identically, every time, without you watching.