A long time ago, people asked how to detect requests in AMF format, or what tools are available to detect them.

Since we are going to talk about the web vulnerability scanner, let's assume that AMF over HTTP is the first one (you don't need to know AMF here, you just need to know that AMF is a data format type).

1. First analyze the AMF format data in http

2. Then fill in the payload in the test parameters

3. Repackaging AMF format data

4. Send HTTP request

1req = {"method": "POST", "url": "http://fatezero.org", "body": "encoded data"}2data = decode_amf(req["body"])3for key, value in data.items():4    d = copy.deepcopy(data)5    d[key] = generate_payload(value)6    body = encode_amf(d)7    requests.request(method=req["method"], url=req["url"], body=body)

There is no problem in the whole process, but if there is another x over HTTP protocol, we have to continue to modify the SQL injection module to support this x protocol. But there are not only SQL injection detection modules in the scanner, but other similar modules. Do I have to change all detection modules every time I add a new protocol?

So we need to separate these protocol parsing and encapsulation into one module.

The pseudo code is as follows:

1 ා utils. Py 2def decode (data): 3 if is AMF (data): 4 data = decode AMF (data) 5 6 if is x (data): 7 data = decode x (data) 8 9 ා recursive decode10 for I in data: 11 data [i] = decode (data [I]) 1213 return data141516 ා detect module. Py17req = {"method": "post",  "url": "http://fatezero.org", "body": "encoded data"}18data = decode(req["body"])19for key, value in data.items():20    d = copy.deepcopy(data)21    d[key] = generate_payload(value)22    body = encode(d)23    requests.request(method=req["method"], url=req["url"], body=body)

The pseudo code is as follows:

1for key, value in x.items(): 2. Data. Reset() 3 x [key] = generate ﹐ payload (value) 4 x.do() ﹐ is responsible for reassembling the data into the original format, and sending 56 ﹐ check according to the original protocol

Because the detection basis of each detection module is roughly several:

• Return content

Return content

• Time based

Time based

• Data of another channel (such as dnslog)

Data of another channel (such as dnslog)

So even if we separate the network operation, it will not affect the detection effect.

When writing the detection module, the writer can not care about what the basic protocol is, how to encode and decode the data, only care about generating the payload according to the value and filling in the corresponding key.

If such a popular coding format appears one day: http://www.a.com/key1, value1, key2, Value2, then all our detection modules need not be modified, just add a set of encode / decode operations on the upper layer. If one day a more popular protocol appears, we only need to provide a set of clients on the upper layer. The work of the detection module is just to generate and fill in the payload.

In 2014, I did a lot of competitive product analysis, including using acenetix WVS, HP webinspect, IBM appscan, netsparker scanning logic of reverse engineering and reverse commerce, as well as reading open-source w3af and arachni code.

If I don't talk about the scanning quality, I only focus on the overall project design and the obscene skills used in the product, then awvs is the most brilliant one. Next, I will introduce the POC classification I learned from awvs in detail.

POC classification:

After acquiring crawler assets and formatting related assets, they will be distributed to different types of POCS for detection. The advantage of this is clear classification, covering most detection stages, and avoiding the need to record additional intermediate state behaviors in order to reduce the distribution of duplicate requests.

Awvs has an interesting function, acumonitor, which is also known as dnslog and anti connect platform. When we saw this function of awvs in 2014, we suggested that wooyun should provide a similar function, namely cloudeye. Tangscan, even though it was the scanner that used this technology earlier in China, of course, there were various projects similar to cloudeye in the future, and naturally there were various scanners using this technology.

But today we are not going to continue to introduce acumonitor, but another interesting feature, acusensor. Acusensor is Iast. As long as you have a little knowledge of Web vulnerability scanner, you should know what Iast does. Then why do I have to carry it out and talk about it alone?

The main reason is that the implementation of acusensor is very interesting. Acusensor provides Java,. Net and PHP, of which the most interesting is the implementation of PHP.

The way to use acusensor in PHP version is to download an ACU phpaspec.php file, and then load the file through auto prepend. As we all know, PHP can't change the built-in function of hook PHP directly. So how can acusensor do the similar Iast function only by relying on a PHP script?

Simply replace all key functions directly. Well, it's really that simple.

Let's introduce this process in detail. In ACU phpaspec.php:

1. Get the file content actually requested by the user

2. Check whether there is a cache. If there is a cache, load and execute the cache directly, and then finish

3. Use token get all to get all tokens, traverse each token, and wrap and replace the functions or statements that you are interested in with your own defined functions

token_get_all

4. Save the replaced content to cache and use Eval to execute

5. Interrupt compilation

__halt_compiler

1<?php23$link = NULL;4$sql = "select * from user where user_id=".$_GET["id"];56mysqli_prepare($link, $sql);

After the conversion of ACU phpaspect.php:

1<?php23$link = NULL; 4$sql = "select * from user where user_id=".$_GET[_AAS91("hello.php", 4, "\$_GET", "id")]; 56_AAS86("hello.php",6,"mysqli_prepare",Array($link, $sql));

The whole process is simple, rough and effective. The advantages of doing so are:

• The implementation is simple, just need to write PHP

The implementation is simple, just need to write PHP

• Simple installation, no need to install the extension, just modify the configuration file

Simple installation, no need to install the extension, just modify the configuration file

• Strong compatibility, easy to be compatible with various environments and versions of PHP

Strong compatibility, easy to be compatible with various environments and versions of PHP

AcuSensor

In the web vulnerability scanner, whether as Party B's commercial product or Party A's self-developed product, speed limit is a crucial function. It can even be said that if your scanner does not have speed limit function, it cannot be used online at all. Next, we will introduce several ways to limit the speed in the scanner.

1. agency

Use the agent as the speed limit function to forward the test traffic of all workers performing scanning tasks to the proxy server:

The proxy server uniformly schedules and sends test request frequency. The advantage of using proxy directly is that it can be compatible with scanners that have not done speed limit function before. The disadvantage is that all time-based tests are invalid (of course, proxy can also be used Return the real response time for judgment, but you still need to modify the detection module), and you are not allowed to add the timeout setting in the detection module.

2. Dual queue

Another method is to use dual queue to realize speed limit function, and the flow chart is as follows:

1. Worker1 gets the task named Target1 from the queue

2. Worker1 gets the task related to Target1 from the Target1 queue

3. By default, tasks related to Target1 are executed in single concurrency. According to the set QPS limit, you can actively sleep or increase concurrency

The disadvantage of this scheme is that it has to be used at the beginning of scanner design. The advantage of this scheme is that each concurrency can keep a stable link with the remote server without affecting the scanning function.

In fact, this section will not talk about a specific vulnerability detection method, but simply talk about what should be done in each stage of the Miss scanning module.

At the beginning of the project, there is no relevant accumulation, so you can choose to look at the awvs detection code. Although it is said that the 10.5 plug-in code is open on the Internet, the plug-in code from 8.0 to 11 is similar to that of 10.5. It is no more than adding a detection module to fix the situation of false or missing reports. You can also look at sqlmap more Code, look at the detection logic, but never learn its code style. From these codes, we can learn many tips, such as dynamic page detection, 404 page recognition and so on. It's easy to understand the relevant logic by looking at the code, but we need to understand why the code handles this way and what the historical background is, so we should use git blast.

If the scanner is an open source project of its own, then it is necessary to appropriately promote its own project, so that more people can use and feedback, and then continue to improve the project, so as to continue to promote its own project, which is a circular process. In a word, improving the accuracy of vulnerability detection requires two conditions: 1. Bad case, 2. Maintenance energy

At the later stage, various conventional vulnerability detection modules have been completed, and there is energy to continuously improve the detection accuracy, and the daily vulnerability POC has also been supplemented by personnel.

Of course, in addition to improving the asset collection, there are also things that help improve the detection effect, such as the above mentioned acusensor, which can achieve the same effect in combination with the rasp within the company, as well as the analysis of access log, database log and other things. Generally speaking, there are no rules and regulations to do leakage scanning, as long as the vulnerability can be found.

AcuSensor

All of the above are related to technology. To do leakage scanning, we need to deal with not only technology, but also detailed and operable vulnerability description and solutions, report quantifiable index data, and most importantly, have a reasonable and convincing throwing pot skill.

These are the tips that I think are useful and can be disclosed in the scanner. I hope they can help you. In addition, if you are interested in vulnerability scanning or IOT automated security products and would like to join us, please send your resume to [email protected].                                                                                                              

MiSRC

Stamp "read the original" to view previous web vulnerability scanner sharing articles