Linux Pipes and Filters Using Tutorial

Occasionally, we can use two commands together, and the output of one command is used as input to another command. This is called a pipe. In order to build a pipeline, you need to use a vertical line between the two commands (| )connection.

Pipe is an important communication mechanism between Linux processes; in addition to pipelines, there are process communication mechanisms such as shared memory, message queues, signals, sockets, and so on.

Pipe using vertical lines (| ) Separate the two commands, and the output of the command to the left of the vertical line is used as the input to the command on the right side of the vertical line. Continuous use of vertical lines indicates that the output of the first command will be used as input to the second command, the output of the second command will be used as input to the third command, and so on.

A tool that accepts data, filters (processes or filters), and then outputs it, called a filter.

grep Command

grep is a powerful text search tool that can use regular expressions and return matching lines with the syntax:
$grep pattern file(s)

>;grep”g/re/p command from ed (a line text editor for Linux), g/re/p is an abbreviation for "globally search for a regular expression and print all lines containing it" It is a global search using regular expressions and prints the matching lines.

A regular expression is a string containing a number of special characters, each of which has a special meaning that can be used to match text. See the regular expression tutorial
for more information.

grep can be thought of as a filter. If you don't specify a file to retrieve for grep, it will be read from a standard input device (usually a keyboard); the same is true for other filters.

The simplest use of the grep command is to retrieve text that contains fixed characters.

For example, using the grep command in a pipe allows only lines containing the specified characters to be output to the display:
$ls -l |  Grep "Aug"-rw-rw-rw- 1 john doc 11008 Aug 6 14:10 ch02-rw-rw-rw- 1 john doc 8515 Aug 6 15:30 ch07-rw-rw-r-- 1 john doc 2488 Aug 15 10:51 intro-rw-rw-r-- 1 carol doc 1605 Aug 23 07:35 The macros$

grep command has many options:

options
Description
-v Reverses the query and outputs unmatched rows. For example, grep -v “test” demo.txt will output a line that does not contain ”test”. -n Output matching rows and line numbers. -l Prints the file name where the matching line is located. -c Outputs the total number of rows matched. -i Matches case insensitive.

The following we use regular expressions to match such lines: contain the characters "carol", then include any number of other characters (including zero), and finally include “Aug”.

Use the -i option for case-insensitive matching:
$ls -l |  Grep -i "carol.*aug"-rw-rw-r-- 1 carol doc 1605 Aug 23 07:35 macros$

sort command

sort command is very useful in Linux, it will The lines in the file are sorted by letter or number. The sort command can get input from a specific file or from stdin.

For example, sort the rows of the foot file:
$sort foodAfghani CuisineBangkok WokBig Apple DeliIsle of JavaMandalaySushi and SashimiSweet ToothTio Pepe's Peppers$

The following options can be used to control the collation:

Options
Description
-n Sort by number, for example, 10 will be followed by 2; the -n option will ignore spaces or tab indentation. -r Sorts in descending order. Sort defaults to ascending sort. -f is not case sensitive. +x Sorts the xth column (starting at 0).

The following example uses the ls, grep, and sort commands in a pipeline to filter the rows containing “Aug” and sort by file size:
$ls -l |  Grep "Aug" |  Sort +4n-rw-rw-r-- 1 carol doc 1605 Aug 23 07:35 macros-rw-rw-r-- 1 john doc 2488 Aug 15 10:51 intro-rw-rw-rw- 1 john doc 8515 Aug 6 15:30 ch07-rw-rw-rw- 1 john doc 11008 Aug 6 14:10 ch02$

The above command sorts the files modified in August in the current directory by size; +4n means the 5th Columns are sorted by number size.

pg and more commands

If there are too many files, all the display will be messy. You can use the pg and more commands to display the page, only one screen at a time.

For example, through the pipeline, use the more command to display the files in the directory:
$ls -l |  Grep "Aug" |  Sort +4n |  More-rw-rw-r-- 1 carol doc 1605 Aug 23 07:35 macros-rw-rw-r-- 1 john doc 2488 Aug 15 10:51 intro-rw-rw-rw- 1 john doc 8515 Aug 6 15:30 ch07-rw-rw-r-- 1 john doc 14827 Aug 9 12:40 ch03...-rw-rw-rw- 1 john doc 16867 Aug 6 15:56 ch05--More--(74% )

As above, only one screen of text is displayed at a time. After the display is full, stop and prompt the percentage of the entire content. Press the space bar to view the next screen. Press b to view the previous screen.