Quickly master awk linux

Basic syntax:

The basic format of the awk instruction is: awk [parameter] & lsquo; awk program & rsquo; file name

The main structure of the awk program is: pattern {action}< Br>

Generally use “relational expressions" as a pattern, for example: $1>4, $2==$3,"banana"~/an/(

This is a regular Matching judgment relationship, if A is a string, B is a positive expression, A~B (determine whether string A contains a substring that can match B expression); A!~B (determine whether string A is Does not contain substrings that match B expressions))

action mainly includes some instructions, such as awk's i/o instructions: print, printf, getline; awk's flow control instructions: if(... ){...}else{...}, do{...}while();

The awk program sequentially searches for each pattern for each input line, for each match. The above mode executes the corresponding action, then reads the next line and starts matching again, knowing that all the input has been processed.

A pattern or action can be omitted in a statement. The default pattern is to match all lines. The default action is to output the current line. Awk reads one line (one record) from the input at a time, and the default line separator is \ . Then awk separates the records into fields, and the default fields are separated into blanks. The first field in a row is $1, followed by $2, $3..., and the entire record is $0.

Awk's built-in variables can be used to get relevant information. Common ones are:

NF(Number of Fields): an integer whose value represents the field contained in $0. number.

NR (Number of Records): is an integer whose value indicates the number of data lines that awk has read.

FILENAME: The name of the data file being processed.

For more information, please refer to http://www.jiayii.com/awk-tutorial/

Basic operation example: test file test.txt, the content is as follows:

(Please note that the blanks may be blank or tab)

1. To get all of John's transcripts, use these commands:

awk '/John/' test.txt (action defaults to output the entire line)

awk '/John/{print $0}' Test.txt

awk '$2=="John"{print $0}' test.txt (default is separated by white space, $2 represents the second field of each line)

Awk '$2~/John/{print $0}' test.txt ($2 and regular expression /John/match)

2. Get a transcript with a score greater than 120

awk '$3>=120' test.txt

3. If the original data file is separated by “, & rdquo;, then all statements can add separator parameters, such as: awk -F", " /John/' test.txt (this is the role of -F)

BEGIN and END

Two special forms in awk that can be used in pattern, providing BEGIN and END to give the program an initial state and execute after the program ends Some finishing work. Any operation listed after BEGIN will be executed before awk starts scanning input, and the operations listed after END will be executed after scanning the full input. Therefore, BEGIN is usually used to display variables and preset initial values, and END is used to output the final result.

Or the above example:

1. Add all of John's scores. The available commands are as follows:

2. The default initial score is 60, and then the scores in the file are accumulated:

Awk array using

aw array using strings as arrays Standard; does not need to be initialized when used;

The actual application array is also the most widely used, we still use test.txt as the data file to calculate the total score of each member:

Practical application

When I use awk, I often get some statistics from the online, and often these log files are not very regular when used, then you can do it first, through grep Or other commands convert the file into a regular output and then use awk for processing.