Author: {JSN} @ arkteam

Original author: he Bo

Research on the key technology of network abnormal behavior detection based on log analysis

Source: he Bo. Research on Key Technologies of network abnormal behavior detection based on log analysis [D]. Civil Aviation University of China, 2016

With the increasing complexity of the network environment, the technology of attack and defense is developing in the competition. In recent years, the intrusion detection system (IDS) has been developing and updating, but there are still high false positive and false negative rates. The research on IDS optimization has never stopped in the academic and industry, among which data mining and association analysis for IDS log information is a very important direction. With the increase of attack methods, the attack process is becoming more and more complex. At this time, the ability to find hidden and complex attacks by simple log analysis of a single device is limited. Especially for an enterprise department with more information systems running at the same time, such as an airport, the log analysis of a single device is very difficult to fully find targeted complex attacks, so it is very necessary for application systems and operations System, firewall, IDS and other system logs are analyzed comprehensively to find more complex and covert attack processes and methods.

This paper proposes an intrusion detection method based on the attack graph, which can take the log information of various devices in the network as the data source of the attack graph, increase the attack scene restoration ability in the process of intrusion detection by modeling the information of each device in a unified way, and show the attack process steps more completely and improve the detection rate of intrusion detection.

1、 Concept and terminology of multi-source attack graph

Attack head node: used to identify the attack process. Each attack head node can uniquely represent an attack process. The structure is as follows.

Warning node: after the warning logs from different devices are collected and normalized, the log information of all different devices can be organized into warning nodes of a unified structure, in which the threat level and threat index of each node are from the device itself generating the warning. The structure is as follows.

2、 Attack graph building process

According to the attack event, multiple warnings from different settings are organized into a complete attack scenario in chronological order. According to the type of attack event, the same type of attack scenario is integrated, so that only one integrated attack process is retained for the same type of attack scenario, so as to determine the minimum threat value required for the establishment of a class of attack process.

For example, an attack type A (worm) has three attack scenarios:

According to the formula proposed in the text, the threat values of the attack head node and warning node are obtained, and then the attack scenarios are fused, and an attack scenario is synthesized as follows:

Finally, through the fusion and summary of multiple attack scenarios, multiple attack scenarios constitute a complete attack graph. The attack graph contains not only the warning types of devices required by each attack scenario, but also the minimum threat values required by such warnings, which helps to shield some false warnings with lower threat values and improve the detection efficiency of intrusion detection.

3、 Attack recognition

The suspicious attack queue is a linked list of warnings from new devices, which can not be determined yet but match some attack scenarios in the attack graph.

Each device has a corresponding suspicious attack queue. When a new warning is generated, it will be compared with the warning chain of the corresponding device in the attack graph. If the graph contains such warning and the threat level meets the minimum requirements, the warning will be added to the suspicious attack queue. Multiple suspicious attack queues will be finally organized into the following suspicious attack queue graph.

The specific attack graph generation algorithm can refer to the original text.

4、 Experimental results and analysis

Most of the existing intrusion detection methods based on attack graph are validated by DARPA's intrusion detection evaluation data set, which contains a complete attack sequence of different types of attacks. However, there is no warning information related to other devices in the network in this data set, and no other device information in other similar data sets. Therefore, in order to carry out the validation of the proposed method Verification, the author built the corresponding experimental environment in the intranet to verify.

During the experiment, common attack tools are used to simulate attacks against some common vulnerabilities that can be exploited in the experimental environment. Through the analysis of all kinds of equipment warnings collected during the attack, 12628 warnings were finally collected. After summarizing according to the attack process, 2181 effective warnings were given for 133 different attacks. After summarizing them into effective attack scenarios, there were 818 warnings, including 362 from IDS. According to 133 attacks, 133 attack scenarios can be initially formed, of which 100 are training sets and the remaining 33 are verification sets.

Take 100 of the above attack scenarios as training sets, and fuse attack scenarios according to 13 common attack types.

Compared with the causal association analysis method and the common attack graph method, this method can present the attack process according to the warning of each device, and restore the attack scene more completely. In order to simulate IDS's false alarm, experiments are carried out under the condition of randomly reducing IDS's false alarm. The first two methods will split the attack scene due to the lack of warning, and can't restore the attack scene well. However, the method proposed in this paper can help to identify the attack process and attack scene when IDS has false alarm because of the use of alarms from multiple device sources And improve the recognition rate of attacks.