How to check HTTP traffic on Linux

Use httpry to sniff HTTP traffic under Linux Tutorial
Sometimes you want to sniff HTTP traffic for some kind of need, then how to check HTTP traffic on Linux? The following is a small series to share the method of detecting HTTP traffic on Linux, I hope to help everyone.

How to detect HTTP traffic on Linux
For example, you might be testing the experimental features of a web server, or debugging a web application or taking full advantage of REST services, or you want to Troubleshoot the PAC (Proxy Auto Configuration) or check for any malware files that were sneaked from a website. Sniffing HTTP traffic can help for whatever reason, for system administrators, developers, and even end users.
Although packet sniffing tools such as tcpdump are widely used for real-time packet dumping, you still need to establish a reasonable filtering mechanism to capture only HTTP traffic; even then, it is usually not easy to interpret them at the HTTP protocol level. The original output. Real-time web server log analysis tools such as ngxtop provide human-readable traces of real-time website traffic, but only if full access to real-time web server logs is available.
Although the sniffing tool like tcpdump is good, it only targets HTTP traffic. In fact, httpry is exactly what we need for an HTTP packet sniffing tool. Httpry captures real-time HTTP packets on the network and displays content at the HTTP protocol level in a human-readable format. In this tutorial we will look at how to use httpry to sniff HTTP traffic.
Installing httpry on Linux
On Debian-based systems (Ubuntu or Linux Mint), httpry does not appear in the base software library. So use its source code to build it:
$ sudo apt-get install gcc make git libpcap0.8-dev $ git clone https://github.com/jbittel/httpry.git $ cd httpry $ make $ sudo Make install
On Fedora, CentOS or RHEL, you can use yum to install httpry as shown below. On CentOS/RHEL, first enable the EPEL software library and then run yum.
$ sudo yum install httpry
If you still want to build httpry using source code on an RPM-based system, it's easy to do this:
$ sudo yum install gcc make git libpcap-devel $ git clone https://github.com/jbittel/httpry.git $ cd httpry $ make $ sudo make install
Basic usage of httpry
The basic usage of httpry is as follows:
$ sudo httpry -i
httpry then listens on a specific network interface and displays captured HTTP requests/responses in real time.

However, in most cases, you will see a fast scrolling output as a large number of packets come in and out. So, you should save the captured HTTP packets for offline analysis. To do this, use the “-b” or “-o” option. The “-b” option allows you to save the original HTTP packet to a binary file and then use httpry to play back the HTTP packet. On the other hand, the “-o” option saves httpry human-readable output to a text file.
I want to save the original HTTP packet to a binary file:
$ sudo httpry -i eth0 -b output.dump
Play back the saved HTTP packet:
$ httpry -r output. Dump
Please note: When you use the “-r” option to read the dump file, you do not need root permissions.
I want to save the output of httpry to a text file:
$ sudo httpry -i eth0 -o output.txt
httpry advanced usage
If you only want to monitor specific HTTP methods (such as GET, POST, PUT, HEAD, CONNECT, etc., you can use the “-m” option:
$ sudo httpry -i eth0 -m get,head

If you downloaded the source code of httpry, You'll notice that the source code comes with a series of Perl scripts that help you analyze the results of the httpry output. These scripts are located in the httpry/scripts/plugins directory. If you want to write a custom script to analyze the output of httpry, these scripts are good examples for reference. Some of these features are as follows:
•hostnames: Displays a list of unique host names and number of hosts.
•find_proxies: Detecting website proxy systems.
•search_terms: Find and count the search terms entered in the search service.
•content_analysis: Find URLs with specific keywords.
•xml_output: Convert the output to XML format.
•log_summary: Generate a log summary.
•db_dump: Dump log file data to a MySQL database.
Before using these scripts, run httpry for a while using the “-o” option. Once you have the output file, run the script once with the following command:
$ cd httpry/scripts $ perl parse_log.pl -d ./plugins
You may encounter warnings for several plugins. For example, if you didn't create a MySQL database with a DBI interface, the db_dump plugin might go wrong. If a plugin fails to initialize, it is automatically disabled. So you can ignore the warning messages.
After parse_log.pl is completed, you will see many analysis results (*.txt/xml) in the httpry/scripts directory. For example, log_summary.txt looks like this:

In a nutshell, httpry can help you if you need to interpret real-time HTTP packets.