Every time we invoke a command, one or more processes are started. Linux must keep track of lots of different programs, all running at the same time. A skilled or well trained system administrator not only needs to create processes, but also be able to keep track of them and send them different types of signals if and when required.
In this guide, we will look at job control and how to monitor processes on Linux systems.
Monitoring Active Processes
At any given time lots of active programs are running on the Linux system. Linux calls each running program a process each process is given a process ID (PID) and manages how the process uses memory and CPU time based on that PID.
When it comes to process monitoring, two invaluable tools used are top and ps commands;
Monitoring Processes with top
Launch your terminal with Ctrl+Alt+T and type top:
$ top
top - 00:28:13 up 14:53, 1 user, load average: 0.25, 0.45, 0.55
Tasks: 232 total, 1 running, 230 sleeping, 1 stopped, 0 zombie
%Cpu(s): 3.4 us, 2.0 sy, 0.0 ni, 94.3 id, 0.3 wa, 0.0 hi, 0.0 si, 0.0 st
MiB Mem : 7861.2 total, 4570.1 free, 1449.7 used, 1841.4 buff/cache
MiB Swap: 2048.0 total, 2048.0 free, 0.0 used. 5912.0 avail Mem
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
2093 frank 20 0 743228 51064 28268 S 4.0 0.6 35:55.56 Xorg
2216 frank 20 0 49964 14016 10828 S 2.3 0.2 18:43.58 i3bar
12586 frank 20 0 2666072 131276 79368 S 1.3 1.6 8:14.45 FoxitReader
1350 mysql 20 0 2003068 358316 34404 S 0.7 4.5 3:35.22 mysqld
2010 frank 9 -11 696116 18308 14088 S 0.7 0.2 4:37.87 pulseaudio
11 root rt 0 0 0 0 S 0.3 0.0 0:03.92 migration/0
296 root 19 -1 82608 41384 39640 S 0.3 0.5 0:08.78 systemd-journal
1155 root 20 0 1062724 10924 9512 S 0.3 0.1 1:26.12 teamviewerd
2210 frank 20 0 21160 6232 5540 S 0.3 0.1 1:51.93 compton
2218 frank 20 0 2504 1840 1656 S 0.3 0.0 1:48.26 i3blocks
3048 frank 20 0 828396 83372 44628 S 0.3 1.0 0:30.43 terminator
4312 frank 20 0 342592 89000 62580 S 0.3 1.1 2:39.50 chrome
8076 frank 20 0 4827320 288404 103744 S 0.3 3.6 21:53.20 chrome
1759295 root 20 0 0 0 0 I 0.3 0.0 0:00.88 kworker/u16:0-events_power_efficient
top allows user interaction. By default, the output is sorted by the percentage of CPU time used by each process in descending order. This behavior can be modified by pressing the following keys from within top:
M: Sort by memory usage.
N: Sort by process ID number.
T: Sort by running time.
P: Sort by percentage of CPU usage.
Note, To switch between descending/ascending order just press R.
Other important keys to interact with top are:
? or h: Help.
k: Kill a process. top will ask for the PID of the process to be killed as well as for the signal to be sent (by default SIGTERM or 15).
r: Change the priority of a process (renice). top will ask you for the nice value. Possible values range from -20 through 19, but only the superuser (root) can set it to a value which is negative or lower than the current one.
u: List processes from a particular user (by default processes from all users are shown).
c: Show programs’ absolute paths and differentiate between userspace processes and kernelspace processes (in square brackets).
V: Forest/hierarchy view of processes.
t and m: Change the look of CPU and memory readings respectively in a four-stage cycle: the first two presses show progress bars, the third hides the bar and the fourth brings it back.
W: Save configuration settings to ~/.toprc.
Detailed output of top
The output of the top is divided into two areas: the summary area and the task area.
The Summary Area in top
It is made up of the the five top rows and gives us the following information:
1.top – 00:28:13 up 14:53, 1 user, load average: 0.25, 0.45, 0.55
- current time (in 24-hour format):
00:28:13 - uptime (how much time the system has been up and running):
up 14:53 - number of users logged in:
1 user - load average of the CPU for the last 1, 5 and 15 minutes, respectively:
load average: 0.25, 0.45, 0.55
2. Tasks: 232 total, 1 running, 230 sleeping, 1 stopped, 0 zombie
- total number of processes in active mode:
232 total - running (those being executed):
1 running - sleeping (those waiting to resume execution):
230 sleeping - stopped (by a job control signal):
1 stopped - zombie (those which have completed execution but are still waiting for their parent process to remove them from the process table):
0 zombie
3. %Cpu(s): 3.4 us, 2.0 sy, 0.0 ni, 94.3 id, 0.3 wa, 0.0 hi, 0.0 si, 0.0 st
- user processes:
3.4 us - system/kernel processes:
2.0 sy - processes set to a nice value — the nicer the value, the lower the priority:
0,0 ni - nothing — idle CPU time:
94.3 id - processes waiting for I/O operations:
0.3 wa - processes serving hardware interrupts — peripherals sending the processor signals that require attention:
0,0 hi - processes serving software interrupts:
0,0 si - processes serving other virtual machine’s tasks in a virtual environment, hence steal time:
0,0 st
4. MiB Mem : 7861.2 total, 4570.1 free, 1449.7 used, 1841.4 buff/cache
- the total amount of memory:
7861.2 total - unused memory:
4570.1 free - memory in use:
1449.7 used - the memory which is buffered and cached to avoid excessive disk access:
1841.4 buff/cache
5. MiB Swap: 2048.0 total, 2048.0 free, 0.0 used. 5912.0 avail Mem
- the total amount of swap space:
2048.0 total - unused swap space:
2048.0 free - swap space in use:
0.0 used - the amount of swap memory that can be allocated to processes without causing more swapping:
5912.0 avail Mem
The Task Area in top
Below the summary area there comes the task area, which includes a series of fields and columns reporting information about the running processes:
PID: Process identifier.
USER: User who issued the command that generated the process.
PR: Priority of process to the kernel.
NI: Nice value of process. Lower values have a higher priority than higher ones.
VIRT: Total amount of memory used by process (including Swap).
RES: RAM memory used by process.
SHR: Shared memory of the process with other processes.
S: Status of process. Values include: S (interruptible sleep — waiting for an event to finish), R (runnable — either executing or in the queue to be executed) or Z (zombie — terminated child processes whose data structures have not yet been removed from the process table).
%CPU: Percentage of CPU used by process.
%MEM: Percentage of RAM used by process, that is, the RES value expressed as a percentage.
TIME+: Total time of activity of process.
COMMAND: Name of command/program which generated the process.
Monitoring Processes with ps
ps command displays snapshot of processes that are currently running on the Linux system. By default, ps command without any arguments displays processes that are associated with the current shell.
$ ps
PID TTY TIME CMD
3204 pts/0 00:00:17 zsh
3316 pts/0 00:00:00 zsh
3577 pts/0 00:00:00 zsh
3578 pts/0 00:00:00 zsh
3614 pts/0 00:00:03 gitstatusd-linu
1609949 pts/0 00:00:00 sleep
1825468 pts/0 00:00:00 top
2089640 pts/0 00:00:00 psFrom the output, we obtain the following information:
- PID The process ID of the process
- TTY The name of the terminal session (shell) that the process is running within
- TIME The amount of CPU time used by the process
- CMD The name of the command that was entered to create the process
ps Option Syntax and Output
ps can accept three different styles: BSD, UNIX and GNU. Let us see how each of these styles would work when reporting information about a particular process ID:
1. BSD
Options do not follow any leading dash:
$ ps p 2117
PID TTY STAT TIME COMMAND
2117 tty2 S+ 0:04 i32. UNIX
Options do follow a leading dash:
$ ps -p 2117
PID TTY TIME CMD
2117 tty2 00:00:04 i33. GNU
Options are followed by double leading dashes:
$ ps --pid 2117
PID TTY TIME CMD
2117 tty2 00:00:04 i3In the above example, ps reports information about the process whose PID is 2117 — in this case, bash. Similarly, you can use ps to search for the processes started by a particular user:
ps U frank(BSD)ps -u frank(UNIX)ps --user frank(GNU)
Now, let us check on the processes started by frank:
$ ps U frank
PID TTY STAT TIME COMMAND
3204 pts/0 Ss 0:19 /usr/bin/zsh
1609949 pts/0 T 0:00 sleep 30
1825468 pts/0 T 0:00 topWe can get the best out of ps by combining some of its options. A very useful command (producing an output similar to that of top) is ps aux (BSD style). In this case, processes from all shells (not only the current one) are shown. The meaning of the switches are the following:
a: Show processes that are attached to a tty or terminal.
u: Display user-oriented format.
x: Show processes that are not attached to a tty or terminal.
Displaying the process with ps in BSD format:
$ ps aux
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
root 1 0.0 0.1 170000 12144 ? Ss Mar26 0:26 /sbin/init splash
root 2 0.0 0.0 0 0 ? S Mar26 0:00 [kthreadd]
root 3 0.0 0.0 0 0 ? I< Mar26 0:00 [rcu_gp]
root 4 0.0 0.0 0 0 ? I< Mar26 0:00 [rcu_par_gp]
root 6 0.0 0.0 0 0 ? I< Mar26 0:00 [kworker/0:0H-kblockd]
frank 2054 0.0 0.4 548668 36536 ? Sl Mar26 0:00 /usr/libexec/goa-daemon
frank 2061 0.0 0.1 317280 8888 ? Sl Mar26 0:00 /usr/libexec/goa-identity-service
frank 2071 0.0 0.0 319252 7548 ? Ssl Mar26 0:03 /usr/libexec/gvfs-afc-volume-monitor
frank 2076 0.0 0.0 238224 5688 ? Ssl Mar26 0:00 /usr/libexec/gvfs-mtp-volume-monitor
frank 2080 0.0 0.0 240632 6332 ? Ssl Mar26 0:00 /usr/libexec/gvfs-gphoto2-volume-monitorWe get the following columns:
USER: Owner of process.
PID: Process identifier.
%CPU: Percentage of CPU used.
%MEM: Percentage of physical memory used.
VSZ: Virtual memory of process in KiB.
RSS: Non-swapped physical memory used by process in KiB.
TT: Terminal (tty) controlling the process.
STAT: Code representing the state of process. Apart from S, R and Z (that we saw when describing the output of top), other possible values include: D (uninterruptible sleep — usually waiting for I/O), T (stopped — normally by a control signal). Some extra modifier include: < (high-priority — not nice to other processes), N (low-priority — nice to other processes), or + (in the foreground process group).
STARTED: Time at which the process started.
TIME: Accumulated CPU time.
COMMAND: Command that started the process.
Running Jobs in the Foreground and Background
Some programs can take a long time to run, and you may not want to tie up the command line interface. Fortunately, there’s a simple solution to that problem: run the program in background mode.
Sending a Job to the Background
Running a program in background mode is very easy, it is done by placing an ampersand symbol (&) at the end of a command. We can demonstrate it using sleep command.
Sending a command to the background via the & symbol.
$ sleep 60 &
[1] 2431805 You can find out about the active jobs (and their status) in your Linux system by running jobs:
$ jobs
[1] + running sleep 60Let us explain the output:
[1]: This number is the job ID and can be used — preceded by a percentage symbol (%) — to change the job status by the fg, bg and kill utilities (as you will be shown later).
+: The plus sign indicates the current, default job (that is, the last one being suspended or sent to the background). The previous job is flagged with a minus sign (-). Any other prior jobs are not flagged.
running: Description of the job status.
sleep 60: The command or job itself.
To display job’s process ID (PID) right before the status, use the -l option.
$ jobs -l
[1] + 2461429 running sleep 60Other options are:
-n: Lists only processes that have changed status since the last notification. Possible status include, Running, Stopped, Terminated or Done.
-p: Lists process IDs.
-r: Lists only running jobs.
-s: Lists only stopped (or suspended) jobs.
When the background process finishes, it may display a message on the terminal like:
[1] + 2461429 done sleep 60This shows the job number and the status of the job (Done), along with the command that ran in the background.
Sending a Running Program to the Background
To send a program running on foreground to background, first stop or pause the process by pressing Ctrl+Z key combination and a job number is assigned . You will then use bg command to send program to the background.
Let’s pause a running job with Ctrl+Z:
$ sleep 3000
^Z
[1] + 2524040 suspended sleep 3000Sending a paused job to the background with the bg command:
$ bg %1
[1] + 2524040 continued sleep 3000You can just check the process if it’s running in the background:
$ jobs -l
[1] + 2524040 running sleep 3000Bringing Jobs to the Foreground
To accomplish this, use the fg command and the background job’s number, preceded by a percent sign (%).
Bringing a background job to the foreground with the fg command:
$ jobs -l
[1] + 3278988 suspended (signal) top
[2] 3280570 running sleep 3000Bringing background job with job ID [2] to foreground with fg command:
$ fg %2
[2] 3280570 running sleep 3000Now, when the job is brought back to the foreground mode it is tied up to the terminal until the program end.
Maintaining a Program Running after Logout
Jobs running are all attached to the session of the user who invoked them. That is, when the session is terminated the jobs are gone too.
it is possible to detach jobs from sessions and have them run even after the session is closed. This is achieved with the nohup (“no hangup”) command.
Syntax:
nohup COMMAND &
Keeping a background job running after log out with the nohup command:
$ nohup sleep 30000 &
[2] 3502697
nohup: ignoring input and appending output to 'nohup.out' Modifying Process Execution Priorities
Sometimes, you may run some applications that need either a higher or lower level of priority. You can modify the priority level assigned by the system to a program using nice and renice commands which allows you to set and change a program’s niceness level.
The nice command allows you to start an application with a non-default niceness level setting.
Syntax:
nice -n VALUE COMMANDAccording to the syntax:
The value parameter is a numeric value from –20 to 19. The lower the number, the higher priority the process receives. The default niceness level is zero.
The COMMAND argument indicates the program started at the specified niceness level.
Modifying an program’s niceness level with the nice command:
$ nice -n 10 sleep 3000You can check the program niceness level using ps command.
$ ps -l 3607152
F S UID PID PPID C PRI NI ADDR SZ WCHAN TTY TIME CMD
0 S 1000 3607152 3527610 0 90 10 - 2652 hrtime pts/0 0:00 sleep 3000You notice that the value in the NI (nice) column is 10.
Changing priority of already running process using renice command.
Syntax:
renice PRIORITY [-p PIDS] [-u USERS] [-g GROUPS]The renice command allows you to change the priority of multiple processes based on a list of PID, USERS and GROUPS.
Changing a running program’s niceness level with the renice command:
$ renice 18 -p 3607152
3607152 (process ID) old priority 10, new priority 18Note: You can increase the priority of a running process to higher level (less than 0) with super user privileges.
$ sudo renice -15 -p 3607152
[sudo] password for frank:
3607152 (process ID) old priority 18, new priority -15Sending Signals to Processes
In Linux, processes communicate with each other using process signals. A process signal is a predefined message that processes recognize and may choose to ignore or act on.
Sending Signals with the kill Command
kill command allows you to send signals to processes based on their process ID (PID). By default, the kill command sends a TERM signal to all the PIDs listed on the command line. You must be the owner of the process or have super user privileges in order to send signal to a process.
Stopping a process with the kill command:
$ ps 3876117
PID TTY STAT TIME COMMAND
3876117 pts/1 SN 0:00 sleep 30000So let’s terminate the above running process with kill command:
$ kill 3876117
[1] + 3876117 terminated sleep 30000 You can see that the process is terminated, and is you check it with the ps command you will obtain no output for the successful terminated process.
$ ps 3876117
PID TTY STAT TIME COMMANDSome processes will ignore the request and you have to force the termination using -s option which allows you to specify other signals (using either their name or signal number). You can also leave off the -s switch and just precede the signal with a dash.
$ ps 3913528
PID TTY STAT TIME COMMAND
3913528 pts/1 T 0:00 sleep 30000Now,let’s terminate the process using kill command:
$ kill 3913528let’s check if the process has been terminated:
$ ps 3913528
PID TTY STAT TIME COMMAND
3913528 pts/1 T 0:00 sleep 30000You find the process did not respond to the signal, so you have to force termination.
$ kill -9 3913528
[1] + 3913528 killed sleep 30000 The process has been killed successfully you can confirm it with ps command.
$ ps 3913528
PID TTY STAT TIME COMMANDkill signal number 9 (KILL) had to be employed to stop the process.
Sending Signals with the killall Command
Using kill command, you can only use the process PID instead of its command name and this makes it difficult to use sometimes. The killall command is a nice solution, because it can select a process based on the command it is executing and send it a signal.
$ ps
PID TTY TIME CMD
3527610 pts/0 00:00:03 zsh
3527686 pts/0 00:00:00 zsh
3527687 pts/0 00:00:00 zsh
3527690 pts/0 00:00:00 zsh
3527695 pts/0 00:00:00 gitstatusd-linu
3987480 pts/0 00:00:00 psNow, let’s kill zsh process with killall command:
$ killall zshwe can confirm if zsh process has been terminated:
$ ps
PID TTY TIME CMD
3527695 pts/0 00:00:00 gitstatusd-linu
3996696 pts/0 00:00:00 psSending Signals with the pkill Command
The pkill command is a powerful way to send processes’ signals using selection criteria other than their PID numbers or commands they are running. You can choose by username, user ID (UID), terminal with which the process is associated, and so on.
The pkill utility works hand-in-hand with the pgrep utility. With pgrep,you can test out your selection criteria prior to sending signals to the selected processes via pkill. the -t option is used on the pgrep utility to see all thePIDs attached to the terminal.
Let’s list the processes ID’s attached to pts/0 terminal:
$ pgrep -t pts/0
4058641
4058652
4058681
4058682
4058684Stopping a group of processes with the pkill command:
$ sudo pkill -t pts/0Now, let’s check if the processes have been killed:
$ pgrep -t pts/0
4058641In the above output there is a process which is still running pid 4058641, let’s examine it:
$ ps 4058641
PID TTY STAT TIME COMMAND
4058641 pts/0 Ss 0:01 /usr/bin/zshwith ps, it is determined that the /usr/bin/zsh program is running on pts/0, which it is supposed to do, because it is our current terminal running.
Conclusion
That’s all about Managing processes on Linux. This guide will help you as a System Administrator to create processes and keep track of them and send them different types of signals if and when required.
You can also check the following articles, very helpful:
