The network operator's five misunderstandings in the log analysis

In the process of using the log, people often face five major misunderstandings. Overcoming these misunderstandings can not only greatly enhance the value of safety facilities, but also resolve potential risks in a timely manner.

In response to emerging security threats, many organizations have deployed multiple security devices. These devices generate a lot of log information. To take advantage of this information, many companies have also deployed log collection and analysis programs. Even so, many users still believe that the role of the security device has not reached the expected value. This happens often because of the five misunderstandings in the log analysis.

Don't View Logs

Many users make a low-level error—don't look at the logs. Although it is important to collect and store logs, it is only necessary to check the logs frequently to understand what is happening in the network environment in order to respond in a timely manner. Once the security device is deployed and the logs are collected, the user needs to continuously monitor it and discover security events that may occur.

Some users only review the logs after major events, although these users can get the benefits of post-mortem analysis, but fail to get the benefits of ex ante prevention. Proactively viewing logs helps users better realize the value of security facilities, understand when attacks will occur, and take action in a timely manner.

Many users always complain that the Intrusion Detection System (IDS) does not work. An important reason for this problem is that IDS often generates false positives that prevent people from taking action based on their warning messages. If people fully correlate IDS logs with other logs (such as firewall logs), they can take full advantage of IDS.

Does not distinguish the priority of the log

The log has been collected, the storage time is long enough, and the log format is also unified, then where should the network administrator start? Users are advised to try to get a high level of summary to see recent security incidents. This requires overcoming another error, that is, not prioritizing the logging. Some network administrators study a large amount of log data without prioritizing the priorities, and the results will be abandoned halfway.

The first step in effective prioritization is to define the strategy. Answering the following questions will help define the strategy: “What are you most worried about?” “Is the attack awkward?” “Has this attack been done before?” Helps users begin to prioritize policies and reduce the burden on users to collect log data every day. .

Log format is not uniform

Log format is not uniform is very common: some are based on simple network management protocols, and some are based on Unix systems. The lack of a unified log format has led companies to require different experts to perform log analysis because not all administrators who are familiar with the Unix log format can understand Windows event logging and vice versa. Most network administrators are usually only familiar with a few systems. Converting the log information generated by the device into a unified format is beneficial for network administrators to perform correlation analysis and decision making.

Log storage time is too short

Many users think that they have all the logs needed for monitoring and investigation, but only after encountering security events, the corresponding log information has been deleted. Security incidents are usually discovered long after an attack or abuse has occurred. If the cost is tight, it is recommended that the user divide the retained log into two parts: short-term online storage and long-term offline storage. Storing old log information on tape saves the cost of offline storage and can be saved for future analysis.

Finding only known bad information

Even the most advanced and most secure users can sometimes get into network traps. This kind of network trap is very sinister and can seriously reduce the value of log analysis. This can happen if the user only looks at known bad information.

The switch is very effective when looking for bad information that has been defined in the log file. However, to fully realize the value of log data, deep digging of logs is required. Without pre-determined bad information, users can find useful information in the log file, including systems that are attacked and infected, new attacks, internal abuse, and intellectual property theft. How can we increase the chances of discovering potential attacks? This requires data mining methods, which enable users to quickly find exception information in the log data.