IMCAFS

 Home

automatic detection of unauthorized vulnerability in zhongtong security open source project

Posted by trammel at 2020-03-15
all

2019 Chapter 10

There are 5286 words in this paper. The reading time is expected to be 14 minutes

Preface

How to realize the automatic detection of security vulnerabilities? I believe this is a topic of common concern for many IT personnel. There are several driving forces behind this: from the perspective of security, the growth of professional penetration testers can't keep up with the growth rate of security testing demand. Moreover, before and after the launch, testing is required, and before and after the subsequent release, it's boring. From the perspective of testing, why can't security testing prepare test cases as well as functional testing or performance testing in advance? After the development is completed, a command is automatically executed by script and the report is output; from the development perspective, now agile, operation and maintenance and development have been gradually integrated, why can't test? In particular, the security test also needs manual detection, which is more time-consuming and seriously affects the efficiency of publishing.

It's true that security testing has its particularity and specialization, and it can't realize fully automatic vulnerability detection at present. However, due to the need of its own security construction, the security team of China Communications is actively exploring local feasible solutions in different links of the development pipeline. There are relevant practices and applications in semi-automatic unauthorized vulnerability detection, although there are still improvements in coverage Space, but it can greatly improve the efficiency of manual security testing, so use this article to share and open source, for the full-automatic ultra vires vulnerability detection scheme in the end of the article, the outlook part is briefly described, and after internal landing, the updated version will be provided in the open source project.

The concept of breach of authority

In the security test, we often encounter various kinds of logic vulnerabilities, and the ultra vires vulnerability is a very common security vulnerability. In OWASP top 10, ultra vires loopholes have been listed for a long time. On the one hand, the harm caused by ultra vires is generally large, such as ultra vires to obtain sensitive information, ultra vires to delete orders of others, ultra vires to add management accounts, etc.; on the other hand, ultra vires loopholes are difficult to detect comprehensively.

Detection of breach of authority

We know that every request received by the server will have its own identity, which will be identified in some place in the request, such as cookie, token, JWT, etc. The ultra vires act can be regarded as a kind of identity to request to obtain the authority which is not owned by it. If it takes effect, it indicates that the request has the ultra vires problem.

At present, the common detection methods for ultra vires are as follows:

Horizontal ultra vires

The a account can obtain (modify, delete, etc.) the B account data by changing an ID or the like in the request.

Vertical ultra vires

Use the account with low authority to send the request that only the account with high authority can have and obtain the operation with high authority.

Unauthorized access

By deleting the authentication information in the request and replaying the request, you can still access or complete the operation.

In order to quickly detect unauthorized vulnerabilities, we have developed a semi-automatic tool to assist. The following describes the design idea of this tool.

System authentication

At present, there are great differences in the implementation methods and protocols of various system authentication. However, there are many schemes to realize automatic system authentication from the perspective of security testing. The common schemes are as follows:

Manual entry of authentication information

After logging in the system to be tested manually, the authentication information, such as token, is obtained directly, and then the authentication information is entered into the unauthorized detection system.

Manual logon

Before the security test, log in at the login interface of the site to be tested by simulating in the ultra vires detection tool, and log in using the ultra vires detection tool to record the authentication information in the whole login process.

automatic logon

With the popularization of unified authentication architecture, it provides very favorable conditions for automatic login. Almost all systems in China Communications use unified authentication for secure access. At this time, the unauthorized inspection tool can be directly connected to the unified authentication system to realize automatic login. In addition, in different authentication systems, for systems with isolation between applications, it is necessary to open specific permissions to support cross application. In the absence of a unified authentication system, the account system and login process of each system are different. To achieve automatic login, a set of login process needs to be prepared for each system. In the face of a large number of systems with different authentication methods, it is difficult to achieve universality.

Traffic acquisition

If you do not access the unified authority management system of the company, you want to obtain all the authority of the system. The usual way is to verify all the functions of the system and then obtain the flow of the function verification. However, there are usually several solutions for the traffic of grab function verification:

Using browser without interface to crawl the site dynamically

Although from Web2.0, the use of the front-end framework and the confusion of JS have increased the complexity of active site crawling, but you can use the browser without interface to dynamically crawl the site. The specific implementation idea of this scheme can refer to the Web2.0 heuristic reptile practice shared by pig man at the white hat meeting of the prophet. See the reference link at the end of the article for details, and there will be no more details here Here we are.

Get all traffic in test / production through traffic gateway

When there is a traffic gateway, the traffic can be grabbed directly from the traffic gateway. Depending on the traffic gateway, for the target system to be tested, you can directly filter out the target requests from the traffic gateway. However, there are some limitations in this scheme. It is still necessary to trigger all functions of the system. If the number of requests obtained is insufficient, it is difficult to cover the whole system.

Passively obtain traffic through agents or plug-ins

After installing plug-ins in the tester's browser and configuring the relevant information of the test system, the tester can manually click the function to be tested to collect the traffic to be tested through plug-ins or proxy servers. Zhongtong's distributed passive scanning system is based on this way to achieve traffic capture. This paper introduces the practice of sharing Zhongtong's distributed passive scanning in detail.

Ultra vires judgement

Whether vertical or horizontal, what we want is actually to send requests that do not belong to a certain identity. Through the first step, we can get the authentication information under different identities. In the second step, we can get the requests to be tested under the site, but we still lack the corresponding relationship between these requests and the permission policy. In this step, different judgment basis can be considered according to different request grabbing methods:

Using browser without interface to crawl the site dynamically

For the vertical ultra vires, if the request can be successfully retrieved, only a subtraction is needed for the request under different identities to get the request that does not belong to that identity. However, some public interfaces still need to be filtered by manual judgment or keyword white list.

It is difficult to judge horizontal ultra vires compared with vertical ultra vires. For some simple sites, we can judge according to the attributes in the form, such as forbidding editing and hiding the form. But for the complex front-end framework, we haven't found a better way. If there is a better way, we hope to share and teach.

Get all the traffic in the test / production through the traffic gateway

The traffic obtained through the traffic gateway is only a simple traffic, and it needs to be replayed after further processing for each request obtained, such as modifying authentication information, etc. by comparing the returned information content before and after replaying, we can judge whether there is any ultra vires.

Passively obtain traffic through agents or plug-ins

In this scheme, the source of traffic is obtained by interaction when the test system clicks the interface, so we can directly judge whether there is the problem of exceeding authority by comparing the returned information of the tester.

Workflow

Workspace configuration process

When preparing for the test, it is necessary to create a workspace for the system to be tested. For the system connected to the unified authentication, the realization of automatic authentication can be considered. For the system not convenient to realize automatic authentication, the system authentication can be realized by manually entering the authentication information.

Figure 1 workspace configuration process

Test flow

During the test, you need to capture the traffic through plug-ins and other forms. At the same time, in order to identify the tester, you need to authenticate the tester, as shown in Figure 2.

Figure 2 test flow

system architecture

The overall architecture of the system is shown in Figure 3, mainly including identity authentication API, traffic transmission device, workspace, scanning task and result display.

Figure 3 system architecture

Authentication API

The unique ID of the current user can be returned through the identity authentication API, and the ID is used to determine the identity of the tester when receiving the traffic from the tester.

Flow transmission device

The traffic transmission device is responsible for transmitting the tester's request to the server of ultra vires detection, which can be implemented in many ways, such as burp plug-in, browser plug-in, proxy server, etc.

working space

Some basic information of the system to be tested will be stored in the workspace, such as the name and Department of the system to be tested. At the same time, it can also be associated with other systems, such as the specific security test application form when we use it. In addition, some configuration information of the system to be tested will also be placed in the workspace, such as the authentication mode, test address and test account to be used of the system to be tested.

Scan task

When the system is started, a thread pool will be opened to consume the scanning tasks in the redis queue. According to the host in the request package and its owner, after some filtering, they flow into the matching workspace. Start different processes according to the configuration of the workspace.

Result display

The scanned results will be stored in the corresponding workspace, and the possible problematic requests will be highlighted during the display. The tester can monitor the scanning results of each request in real time, and perform filtering, filtering, replay and other operations as required.

Effect demonstration

Before the test, the workspace needs to be configured. Here, the unified authentication system is used as a demonstration.

Figure 4 workspace configuration

After the configuration is completed, the test can be started and the test flow can be observed in real time. And according to the needs of packet comparison and request replay.

Figure 7 request replay

reference material:

OWASP TOP10 ：https://www.owasp.org/images/7/72/OWASP_Top_10-2017_(en).pdf.pdf

Web2.0 heuristics: https://xzfile.aliyuncs.com/upload/zcon/2018/11/u Web2.0 heuristics: pig man.pdf

Author brief introduction

R00, information security engineer of China express, mainly responsible for daily risk assessment and security testing.

Welcome to Zhongtong SRC article exchange group

Zero distance communication with the author (if the QR code fails, please add Chinacom SRC operation wechat: sunandcc1022)

Pay attention to us, leave messages in the background that you are interested in, and follow-up articles will provide more highlights

Team Introduction

Zhongtong information security team is a young, upward, down-to-earth family striving for dreams. Our goal is to build a fully automatic information security intelligent perception response system and management operation platform based on massive data. We are committed to supporting the safe development of the whole ecological chain business (express, express, e-commerce, media, finance, aviation, etc.) of China Express Group. Our technical route follows the development of the industry, from big front-end to cloud native, from big data to machine learning, from devosecops to zero trust security architecture. The data scale of the largest express delivery company in the world will also be a very big challenge. Our focus is not only on domestic first-line Internet companies, but also on Google, Facebook, Amazon and other practices in basic security and data security.

Join us

If you are interested in our team or what we do, and hope to make achievements in the field of engineering technology, you are welcome to join us. We need talents in information security, distributed platform development, big data, risk control, product, operation, etc. base Shanghai, working place: Hongqiao Vanke center and Zhongtong headquarters. Resume delivery address: [email protected]

© 2023