Awk command is the most powerful scripting language in Linux that is developed with the purpose of text processing and transforming text in a way like producing formatted reports. It is relatable to grep and sed commands as it acts as filters. AWK command doesn’t have any specific definition to its name as it is named after the surname of its original developers Alfred Aho, Peter J. Weinberger, and Brian Kernighan.
In this article we will cover 14 awk commands which might be useful. In this context, I have the following set of data in the sample.txt file in my system.
1 ) Using Variable
Suppose you want to only print specific field data from the data set. For this case, the awk command will handle it efficiently; you simply need to execute the command in the following way.
Note: $n will print the value nth field where n denotes the number.
$ awk '{print $2}' sample.txt
2) File Separator
We can separate text using the awk command using the -F flag followed by the separator. By default, the awk command will separate the line by space and tab space. Let’s take a look at how text is separated in the following example.
$ awk -F':' '{print $1}' /etc/passwd
3) Awk Preprocessing
Suppose you are viewing a set of specific field data using the awk command and you want to add a header or title to the result. For this, we use BEGIN keyword which is executed the script before data processing. To achieve it we need to execute the command in the following way.
$ awk -F':' 'BEGIN {print "Users were"} {print $1}' /etc/passwd
4) Awk Post processing
For this, you need to use the END variable to run the script after data processing. we need to execute the script in the following way to achieve the expected result.
$ awk '{print $2} END {print "Thank You."}' sample.txt
5) Running Script From File
First, let’s create a new script file that contains our script.
$ touch myscript
After the file is created let’s write the script. Simply copy-paste the following script.
$ vi myscript
BEGIN {
print "All Users List"
}
{
print $1
}
END {
print "The end"
}
In the above script, FS will separate the text according to the provided value. Now we can run the script using the -f flag similar to the example given below.
$ awk -f myscript sample.txt
6) Using Multiple Command
Multiple commands are separated by the semicolon in the script. In the following example, we separate two commands using a semicolon. In command, we override the inbuilt variable with a new value and print it.
$ echo "Good Morning sir" | awk '{$2="Afternoon"; print $0}'
7) Allocating Line Numbers to Awk Command Output
Awk command checks the text file thoroughly each line and using NR variable in the script will print the line number along with the output. In the following example, we have printed all system users along with their line numbers.
$ awk -F':' '{print NR "--" $1}' /etc/passwd
8) Count Number of Fields
In some cases, while reviewing data from the data set some fields maybe forgot to insert or added extra field data. The counting field will be more reliable than reviewing every field manually. To achieve this you need to use the NF variable in the script.
$ awk '{print $0 " Count=" NF}' sample.txt
9) User Defined Variable
In awk scripting, we can define a variable and assign value to it. Let’s look up examples of how it should be done.
$ awk '
BEGIN{
demo="Hello World"
print demo
}'
9) If condition in awk
We can imply if condition in the script to filter the certain criteria matching data. For example, in system user data you want to list out all even line users using the awk command to do so, we need to execute the command in the following way.
$ awk '
BEGIN{
print "List Of All Even Numbered Line User"
FS=":"
}
{
if(NR%2==0){
print $1
}
}' /etc/passwd
11) For Loop in AWK
We can use for loop in many ways but in the example below for loop is used to print random numbers from 1 to 100 ten times. To generate a random number we will use the rand() which is an inbuilt function that provides random numbers. We can limit the random number from its pool as they are in decimal. We need to multiply it by 100 and convert it to an integer.
$ awk 'BEGIN {
for (i = 1; i <= 10; i++){
print int(100 * rand())
}
}'
12 ) While Loop in Awk
While Loop keeps on Iterating until the condition matches. Let’s add up all the numbers from 1 to 10 using a while loop.
$ awk 'BEGIN {
sum = 0;
i = 1;
while(i<=10){
sum += i
i++;
}
print sum
}'
13) User Defined Function
We can create the function and call it in the following way.
$ awk '
BEGIN{myfunc()}
function myfunc(){
print "The Function has been called."
}'
14) Mathematical Function in Awk
Awk also provide mathematical function like
sin(x) | cos(x) | sqrt(x) | exp(x) | log(x) | rand()
We can use these functions to get respective values while doing mathematical calculations. In the example below I have print the exponential of 5 likewise you can try other functions.
$ awk 'BEGIN{x=exp(5); print x}'
Conclusion
In this article we learned about awk command and how to script in using awk command. We have covered 14 useful awk command examples which might be useful in our daily routine of Linux operations and development.
Also Read : 15 Quick Wget Command Examples in Linux