With the development of Internet security vulnerabilities, Java deserialization vulnerabilities, struts command execution vulnerabilities, ImageMagick command execution vulnerabilities and other high-risk vulnerabilities frequently break out. In this case, in order to quickly form the vulnerability detection capability after the vulnerability outbreak, and at the same time conduct comprehensive and fast security detection on the website or host, the tiesec security team (http://www.tiesec.net) has developed a set of simple and easy-to-use distributed web vulnerability detection system wdscanner.
At present, wdscanner platform has the following functions:
Customer management: add customer, add customer holding system, set scan strategy of customer holding system, periodic scan, etc
Vulnerability scanning: Distributed Web scanning, periodic vulnerability scanning, sensitive information disclosure scanning, vulnerability management, etc
Information collection: website title, banner, port opening, sub domain name, CMS fingerprint, operating system version, development language, WAF, CDN, middleware, etc
Website crawler: website link crawling, dark chain detection, bad chain detection, sensitive word detection
Other functions: agent collection, retrieval center, Export Report
This platform only collects the information of the target system and detects the openness of the port. The vulnerability scanning calls the scanning core of WVS, which is mainly for the convenience of everyone to carry out the security detection of the target system and issue the Chinese report.
The detection of any target on the Internet can only be carried out after obtaining authorization. If you need to test the usability of this platform, please set up the target environment by yourself. It is hereby declared that the user shall be responsible for the consequences of his / her acts endangering the network security, which is irrelevant to the author.
V1.0 version: in March 2019, it was open-source on GitHub, but I didn't expect that you would pay so much attention to this small platform before, so the installation methods provided are also relatively simple, causing many friends to encounter some problems during installation and waste some time. I'm sorry.
V1.1 version: in May 2019, according to your feedback, the whole environment was repackaged and three detailed installation methods were provided. Because some modules are connected to other systems in our intranet, we will not provide them here.
GitHub address: https://github.com/tidesec/wdscanner/
https://github.com/TideSec/WDScanner/
Mode 1: virtual machine environment
1. Download virtual machine image
The packed VMware image is made with VMware Workstation 15 pro, downloaded directly, decompressed, and opened with VMware. In theory, the version above 15.0 can be used.
In VMware Workstation 15 and above, if you prompt "import failed" and "specification consistency and hardware compliance", click "retry" directly.
Link: https://pan.baidu.com/s/1lqglkwzknajeyrenaxlja extraction code: icmv
The virtual machine is a win7 system with user name of tidesec and password of 123456. By default, the virtual machine is connected to the network in NAT mode. In theory, the IP address can be automatically assigned and can be used directly.
In addition, the virtual machine is win7 SP1, no patch has been made, and ms17010 or something should also be available. If necessary, please reinforce the virtual machine by yourself.
2. Run phpstudy on the desktop to start the service.
3. Run the file of tide-proxy-bat.bat and tide-wdscanner-bat.bat on the desktop.
Tide-proxy-bat.bat
Tide-WDScanner-bat.bat
4. This machine accesses http://127.0.0.1, the user name and password is admin / 123456, log in, then add customers, add tasks, perform scanning, etc.
Mode 2: semi integrated installation
1. Download package
The phpstudy environment is packed and unzipped to the root directory of disk C. The directory name should not change, that is, the directory of C: \ wdscanner.
2. Install python2.7, ruby, nmap, awvs10.5, Pip, etc.
Install the corresponding software from the wdscan soft directory of the installation package to configure the environment variables.
3. Install Python dependency Library
Because the background scripts are all run in Python, some third-party libraries are called, and there is a requirements.txt file in the taskpython directory
Execute PIP install - R requirements.txt in the taskpython directory.
pip install -r requirements.txt
4. In the taskpython directory, execute two files, tide-proxy-bat.bat and tide-wdscanner-bat.bat, respectively, to open the wdscanner background task and the agent collection task.
Tide-proxy-bat.bat
Tide-WDScanner-bat.bat
5. This machine accesses http://127.0.0.1, the user name and password is admin / 123456, log in, then add customers, add tasks, perform scanning, etc.
Mode 3: manual installation
1. Install python2.7, ruby, nmap, awvs10.5, Pip, etc.
It is recommended to use Windows environment, because WVS can only run in Windows environment at that time (now it seems that Linux version is available), and WVS is recommended to use wvs10.5 version. After Python and pip are installed, configure the environment variables.
I packaged several software needed above, installed them one by one in wdscan soft directory, download address:
wdscan-soft
2. Install PHP running environment
It is recommended to use Apache, PHP version 5. * not too high, phpstudy is recommended, simple and convenient, one click deployment.
Place all files in the C: \ wdscanner \ www directory. Because of some laziness, write some absolute paths. If you have the ability to edit the code, it is strongly recommended to develop it by yourself.
3. Install Python dependency Library
Because the background scripts are all run in Python, some third-party libraries are called, and there is a requirements.txt file in the taskpython directory
Execute PIP install - R requirements.txt in the taskpython directory.
pip install -r requirements.txt
4. Extract wdscanner.sql.zip, create a new database wdscan in mysql, import wdscanner.sql, and modify the database password in config.inc.php under the include directory.
5. Run taskscan.py, taskspider.py and taskinfo.py under the taskpython directory on the scan node to respectively scan tasks, analyze website crawling keywords, collect information, etc. (w3af is not integrated in this version due to its cumbersome deployment)
6. Execute IP pool.py and asset quality.py under the taskpython / proxy directory to collect and rate the agents.
7. This machine accesses http://127.0.0.1, the user name and password is admin / 123456, log in, then add customers, add tasks, perform scanning, etc.
1. Login interface
The color of the project hall is quite vulgar
2. Distributed scanning
Wdscanner uses distributed web vulnerability scanning technology. The front-end server interacts with users and issues tasks. It can deploy multiple scanning node servers to complete scanning tasks more quickly.
Because the results of WVS are in English, the friendliness is not high if it needs to provide reports to customers, so the scanning results, reinforcement suggestions, vulnerability description, etc. of WVS are localized.
The Chinese version of WVS mainly crawls the official vulnerability database of WVS, manually translates the more common vulnerabilities, and then uses Google translation to translate other vulnerabilities, manually checks them, and finally contains about 670 vulnerabilities.
The Sinicized database can be obtained from this table, https://github.com/tidesec/wdscanner, in the vul? CN table of the database file.
https://github.com/TideSec/WDScanner
Secscanner + w3af + awvs (secscanner is another set of Web scanners under construction, w3af is the best open-source scanner) are used in the scanning core library. More scanning tools may reduce the scanning speed, but the false alarm rate will also be greatly reduced, and the most practical scanning strategy can be selected to save time.
3. Customer management
Be able to manage customers and assets, customize scanning and monitoring schemes according to customers' needs, regularly scan and crawl websites, retrieve threats such as sensitive words, bad chains, dark chains and information leakage, and timely remind and inform customers of risks found.
4. Website information collection
After adding new tasks, the background can actively identify the target banner and operating system information, port opening, sensitive file scanning, etc.
Automatically identify development language, WAF, CMS and middleware, scan common ports and judge their services.
The collection of subdomains uses two methods: violent enumeration and Internet search, which ensures the availability of subdomains and shortens the search time.
5. Website crawler
At present, government websites are sensitive to dark chain, sensitive words and bad chain, and website crawlers can better solve this part of the demand.
The website crawler mainly crawls the whole site page of the target periodically. The crawler algorithm mainly adopts the breadth first traversal strategy, which can collect web links, dynamic URL, sensitive words of the website, dark chain, bad chain detection, store web snapshot, etc.
6. Special test
Wdscanner integrates special vulnerability detection function, which can quickly deploy detection POC in case of high-risk vulnerabilities and conduct batch security detection on customer websites.
7. Search center
Search center can use keywords to search vulnerability scanning, information collection, website crawler, etc., such as vulnerability type, operating system type, open port, middleware type, development technology, etc.
Website URL retrieval, for example, to retrieve the URL containing. Action.
8. Agent resource pool
The built-in proxy resource pool function can dynamically score and sort the collected proxy addresses, and can intelligently switch IP addresses when scanning and detecting the blocked IP addresses.
9. Node management
Scan nodes are managed. Nodes not in the scope cannot request platform tasks.
10. Report output
The report output is an indispensable part of the professional scanner. It seems that the function is not very impressive, but it does take us a lot of time and energy to achieve this function. Now it only implements a regular report template, and there are some repetitions in vulnerability classification, which will be gradually improved in the future.
In task management, each task can be exported. You can see the general layout and content of the report. I wanted to add a chart, but I can't do it for the time being. The generated report is roughly the same.
Overview:
Vulnerability display:
Sensitive words:
Information disclosure:
1. The platform was developed before. Some code may have bugs. Do not deploy it on the Internet;
2. Some functions have been commented out in the code due to their long time consumption. It is recommended to develop them again if you are interested.
3. The virtual machine image provided is win7 SP1, no patch has been applied, and ms17010 or something should also be available. If necessary, please reinforce the virtual machine by yourself.