What is the difference between Grep, Awk and Sed commands in Linux?

Grep command in used for finding particular patterns in files and outputs all the result containing the search pattern. Awk on the other hand is also used for searching a file for certain patterns but goes ahead to perform a certain task on pattern match. Grep and awk can be used at the same time to narrow down the search enhance results. Grep is a simple tool to use to quickly search for matching patterns but awk is more of a programming language which processes a file and produces an output depending on the input values. Sed command is mostly useful for modifying files. It searches for matching patterns and replaces them and outputs the result.

Using Grep command in Linux

Grep command is used for finding particular patterns in a file and to display all the fields matching that pattern. The searched pattern is what is normally referred to as regular expression.

In my examples, I have a file called test as below:

$ cat test
Andy wendie account 45000 
Abon vicky account 25000 
varun manager sales 50000 
Manny manager account 47000 
tarun cindy sales 15000 
Naom clerk sales 23000 
sunil cindy sales 13000 
satvik director purchase 80000 

To output every field containing the word ‘manager’ in a file:

$ grep -i "manager" test 
varun manager sales 50000 
Manny manager account 47000

The ‘-i’ in the above command tells the grep command to ignore any case sensitivity.

‘-c’ outputs the count for the number of occurrences of the matched pattern in a file.

$ grep -c "manager" test 
2

To search from all the files in a directory containing a certain pattern, we use ‘*’. It shows the specific files and the results of the match

$ grep "manager" * 
test:varun manager sales 50000 
test:Manny manager account 47000

To only output the files containing the matched pattern we use ‘-l’

grep -l "manager" * 
test

You can also specify file names if you know them to output all the matches.

$ grep “manager” test test1 test2

Match whole words in a file. Grep by default outputs all the occurrences of a certain pattern even if it is found in substring. To match whole words only, we use ‘-w’ flag

$ grep -w "manager" test 
varun manager sales 50000 
Manny manager account 47000

To include sub-directories in the search, use ‘-r’ as below:

grep -r “manager” *

Output matched pattern only. Grep displays the whole sentence containing the matched pattern. We can display the search pattern only by using ‘-o’ string.

$ grep -o "manager" test 
manager 
manager

Output with numbers are the matched patterns using ‘-n’. This displays the specific number count where the matched pattern is in the file.

$ grep -n "manager" test  
3:varun manager sales 50000 
4:Manny manager account 47000

Invert the match to display output not matching the input pattern using ‘-v’. In this case, we are outputting anything else that does not contain the word ‘manager’.

$ grep -v "manager" test  
Andy wendie account 45000 
Abon vicky account 25000 
tarun cindy sales 15000 
Naom clerk sales 23000 

Match all fields starting with a certain string, for example, output all fields staring with the word ‘manager’

grep "^manager" test

Match lines ending with a given string. The ‘$’ regular expression signifies the end of a line and can be used to match lines ending with a specific string. In our case, we watch to match lines ending with ‘0’.

$ grep "0$" test 
Andy wendie account 45000 
Abon vicky account 25000 
varun manager sales 50000 
Manny manager account 47000 
tarun cindy sales 15000 

To display the number of lines before and after a match

• -A – specify number of lines to display after a match
• -B – specify number of lines to display before a match
• -C – specify number of lines to display before and after a match

Use ‘-e’ to specify multiple match patterns in the same line

grep -e “manager” -e “ Naom” -e “varun” test

To limit grep output to a number of lines, we use ‘-m’

$ grep -m1 "manager" test 
varun manager sales 50000
grep -m2 "manager" test  
varun manager sales 50000 
Manny manager account 47000

Using AWK Command in Linux

Awk command is more of scripting language used in manipulating data and generating reports. It does not require any compiling and a user can use numeric functions, variables, string functions, and logical operators. It enables writing of simple and effective programs in statement forms to search through a file for a specific pattern and performs an action when a match is found.

When using ‘awk’ we enclose patterns in curly braces. Both pattern and action form a rule and the entire awk program is enclosed in single quotes.

How to use awk command in Linux

Awk command is used to print out the content of a file by default. In this case, no pattern is specified and therefore action applies to every line of the file.

$ awk ‘{print}’ test
Andy wendie account 45000 
Abon vicky account 25000 
varun manager sales 50000 
Manny manager account 47000 
tarun cindy sales 15000 

To print a line matching a given pattern, for example, let us print all lines having the word ‘manager’ in them.

$ awk ‘/manager/ {print}’ test
varun manager sales 50000 
Manny manager account 47000

In the case above, every line with the word ‘manager’ will be printed. We can specify that we only want to print those lines which start with manager. This is accomplished using the line token ‘^’ as below:

awk ^manager/ {print}’ test

Awk is used to split a line into fields and print out the content. Let’s print the first and third parts of the lines in test.

$ awk ‘{print $1,$3}’ test
Abon account 
varun sales 
Manny account 
tarun sales 

Note that $0 outputs the entire line.

NR – Prints lines along with the line numbers.

$ awk '{print NR,$0}' test 
1 Andy wendie account 45000 
2 Abon vicky account 25000 
3 varun manager sales 50000 

Can also be used to specify to print from a certain number to another. Let us display from line number 3 to number 6.

$ awk ‘NR==3, NR==6 {print NR,$0}’ test

NF – outputs the number of fields contained in each record (line)

$ awk '{print NF}' test  
4 
4 
4 

$NF – prints the last columns

$ awk '{print $NF}' test    
45000 
25000 
50000 

OFS – Output field separator – used to specify the separator for the output values.

$ awk 'OFS="/" {print $1,$4}' test   
Andy/45000 
Abon/25000 
varun/50000 

Awk BEGIN and END rules

BEGIN rule is expected once before any text processing and is executed first before anything else.

$ awk 'BEGIN {print "my test file"} {print $0}' test 
my test file 
Andy wendie account 45000 
Abon vicky account 25000 
varun manager sales 50000 

Adding patterns

Output for a value greater than. For example, in our file the last field represent salary figures, we need to output only when the value is greater than 25,000

$ awk '$NF>=25000 {print $1,$4}' test 
Andy 45000 
Abon 25000 
varun 50000 

Built-in functions

awk comes with built-in functions such as numerical as demonstrated below

$ awk '{ print sqrt(625)}' 
25

Awk Scripts

If you find yourself writing complicated awk commands, you can opt to put them in a script file (.awk) and run them from there.

Using SED command in Linux

SED is short for stream editor. It can be used to perform different functions such as searching, find and replace, insert and delete. It is, however, common for find and replace and you don’t have to open the file to substitute words.

To replace the word ‘manager’ with ‘operations’ in our test file and output the result:

$ sed 's/manager/operations/' test 
Andy wendie account 45000 
Abon vicky account 25000 
varun operations sales 50000 
Manny operations account 47000 

To substitute every occurrence, we include ‘g’ at the end of the expression as below

$ sed 's/manager/operations/g' test 
Andy wendie account 45000 
Abon vicky account 25000 
varun operations sales 50000 
Manny operations account 47000 

To replace only a specific number of occurrences, specify the number in place of ‘g’. For example, to replace up to the fourth line:

sed 's/manager/operations/4' test 

To replace from a certain occurrence to the rest of the file, specify the number to start from together with ‘g’ to signify all or the remaining part of the file.

sed 's/manager/operations/4g' test 

To replace only on a specific line, specify the file line as below where we are replacing on the third line.

sed '3 s/manager/operations/' test 

To print the replaced line only instead of printing all the file content:

sed -n 's/manager/operations/p' test

To replace from a certain line onwards, for example, replace from the 3^rd to the end:

sed '3,g s/manager/operations/' test 

Use SED to delete lines of a file. For example, to delete the 5^th line of a file:

sed '5d' filename.txt

To delete the last line:

sed '$d' filename.txt

To delete from nth to the last, for example, from 3^rd to the last:

sed '3,$d' filename.txt

To delete a pattern matching file:

sed '/pattern/d' filename.txt

This has been our guide on how to use Grep, Awk and Sed commands in Linux. I hope it has been informative. Check more interesting Linux guides below:

• Using TLDR – Best Linux Command Line Help Pages and Cheats
• Using tmux on Linux and macOS – tmux cheat sheet
• Install XCode Command Line Tools on macOS