Linux Process Management: ps, top, kill and Signals
Find a process, read its state, and stop it cleanly. Learn the process tree, the signals, and the tools that show what is running.
By Velloreth
What a process is
A process is a running program with its own memory, its own file descriptors, and a process ID. Every process except the first has a parent, and the parent-child relationship forms a tree.
ps -ef
# UID PID PPID C STIME TTY TIME CMD
# root 1 0 0 10:00 ? 00:00:01 /sbin/init
# root 412 1 0 10:00 ? 00:00:00 /usr/sbin/sshd
# user 900 412 0 10:05 pts/0 00:00:00 -bash
# user 950 900 0 10:06 pts/0 00:00:00 ps -ef
PID is the process ID. PPID is the parent’s PID. CMD is the command.
The tree matters because killing a parent can orphan its children, and a shell that exits sends a hangup to its jobs.
Listing processes
ps shows a snapshot.
ps aux # every process, BSD style
ps -ef # every process, System V style
ps -ef --forest # as a tree
ps -u alice # only alice's processes
top shows a live view, sorted by CPU by default.
top
# Press M to sort by memory, P by CPU, q to quit
htop is a friendlier interactive version, if it is installed.
pgrep and pkill match by name.
pgrep -a nginx # show the PID and the command line
pkill -f "node server" # kill by a full command line match
pidof returns the PID of a named program.
pidof nginx
Process states
| State | Meaning |
|---|---|
| R | running or runnable |
| S | sleeping, interruptible |
| D | sleeping, uninterruptible (usually I/O) |
| T | stopped |
| Z | zombie |
A D state is worth attention. It means the process is waiting on I/O and cannot be interrupted, and a process stuck in D often indicates a disk or a network filesystem problem.
A Z state is a zombie. It is a finished process whose parent has not read its exit status. It holds a process table slot and nothing else.
Signals
A signal is an asynchronous notification to a process. The common ones:
| Signal | Number | Meaning |
|---|---|---|
| SIGHUP | 1 | hangup; often means reload the config |
| SIGINT | 2 | interrupt, from Ctrl+C |
| SIGTERM | 15 | terminate, the polite request |
| SIGKILL | 9 | kill, cannot be caught |
| SIGSTOP | 19 | stop, cannot be caught |
| SIGCONT | 18 | continue a stopped process |
SIGTERM is the default for kill. It asks the process to stop, and a well-written process catches it, finishes the current work, closes its connections, and exits.
SIGKILL cannot be caught. The kernel stops the process immediately. The process cannot clean up, so a temporary file may be left behind and a connection may be left open.
Always try SIGTERM first.
kill 1234 # SIGTERM
sleep 5
kill -9 1234 # SIGKILL if it is still there
SIGHUP is often a reload. Many daemons catch it and re-read their configuration without restarting, which avoids dropping connections.
kill -HUP $(pidof nginx)
Finding what holds a port
ss -tlnp
# State Recv-Q Send-Q Local Address:Port Process
# LISTEN 0 511 0.0.0.0:80 users:(("nginx",pid=1234,fd=6))
ss is the modern replacement for netstat. The flags are -t for TCP, -l for listening, -n for numeric, and -p for the process.
lsof is the alternative.
lsof -i :8080
Both need elevated privileges to see a process owned by another user.
The load average
uptime
# 10:30:00 up 2 days, 3:14, 2 users, load average: 1.20, 0.95, 0.80
The three numbers are the 1, 5 and 15 minute averages. They count processes that are running or waiting for a resource.
Compare to the CPU count.
nproc
# 4
A load of 4 on a 4-core machine is fully utilised. A load of 8 means processes are waiting, and the cause may be CPU, disk or a lock.
A high load with low CPU usage is usually I/O. Check the disk with iostat or the wait time in top.
Background and foreground
long_command & # run in the background
jobs # list the shell's jobs
fg %1 # bring job 1 to the foreground
bg %1 # resume job 1 in the background
A background job still dies when the shell exits, because the shell sends a hangup. To keep it running:
nohup long_command &
disown %1
A terminal multiplexer is the robust approach. tmux or screen keeps the session alive on the server, so a dropped connection does not kill the process.
tmux new -s work
# Ctrl+B, D to detach
tmux attach -t work
Managing a service
systemd is the init system on most modern distributions.
systemctl status nginx
systemctl restart nginx
systemctl enable nginx # start at boot
journalctl -u nginx -f # follow the logs
systemctl manages the service, and journalctl reads its logs. A service that fails to start usually has the reason in the journal.
A process investigation
When a service is slow or unresponsive, the process tools give a short sequence that usually finds the cause. The first step is to check the load average against the CPU count, which says whether the machine is saturated. The second is to sort the processes by CPU and by memory, which identifies the one that is consuming the resource. The third is to check the process state, because a process in the uninterruptible state is waiting on I/O rather than on the CPU, and the fix is at the storage layer rather than in the application. The fourth is to check the open files and the connections, because a process that has exhausted its file descriptors behaves like a process that is out of memory. The fifth is to read the service’s own log, because the process tools show the symptom and the application log shows the cause. Working through that sequence takes a few minutes and resolves the large majority of a performance complaint.
Common mistakes
kill -9 as the first attempt. The process cannot clean up, which can leave a lock file or a corrupt state. Try SIGTERM first.
Killing a parent and expecting the children to stop. They are reparented to init and keep running. Kill the process group with a negative PID, or use the service manager.
A zombie that is blamed for a resource problem. A zombie holds only a process table slot. The real problem is the parent that is not reaping.
A high load average read as a CPU problem. It counts processes waiting on I/O too. Check the disk.
A background job that dies when the shell closes. Use nohup, disown, or a multiplexer.
netstat on a system where it is deprecated. Use ss.
A process stuck in D that is killed repeatedly. It cannot be killed until the I/O completes or times out. The fix is at the storage layer.
A quotable summary
A Linux process has a PID, a parent and a state, and the parent-child relationship forms a tree. ps, top and ss show what is running and what holds a port. SIGTERM asks a process to stop and lets it clean up, while SIGKILL cannot be caught and stops it immediately, so SIGTERM should always be tried first. A zombie is a finished process whose parent has not read its exit status, and a load average above the CPU count means processes are waiting, which may be a CPU or an I/O problem.
Related reading
For the shell around this, see linux-command-line-basics and for the permissions, linux-file-permissions. For the broader context, bash-scripting-basics and incident-response-solo-developers.