IMCAFS

 Home

introduction of code security analysis software in the industry

Posted by punzalan at 2020-03-11
all

Preface

The types of application security analysis are divided into four directions according to the usage scenarios:

Static ast (SAST) technology usually analyzes the application's source code, byte code or binary code in the programming and / or testing software life cycle (SLC) stage to find security vulnerabilities.

Dynamic ast (DAST) technology analyzes the dynamic running state of an application during testing or running. It simulates attacks against applications, usually web enabled applications and services, analyzes the application's response to determine whether it is vulnerable.

Interactive ast (IAST) technology combines the elements of SAST and DAST. It is often implemented as a proxy in a test run-time environment (for example, to test a Java virtual machine [JVM] or. Net CLR) to observe operations or attacks and identify vulnerabilities. (it can be found that Iast is similar to rasp, which can be combined with scanner to integrate security detection into products, and judge the authenticity of vulnerability through a mechanism similar to piling.)

Mobile ast performs SAST, DAST, Iast and / or behavior analysis on byte or binary code to identify vulnerabilities in mobile applications.

At present, the better cases of static code audit are the static check combination of sonarlint + findbugs + Android lint which is modified in Android. But for the server-side code quality and security aspects of the detection means is still seriously inadequate. The current open source tools are generally applicable to code detection rather than security detection, and the detection focuses on bugs rather than vulnerabilities.

The software involved in this analysis includes the following tools

Https://github.com/google/shipshape this is a static program analysis platform published by Google. It is written in go language and supports JS, Java and python to run in docker image mode. You can use go language and Java (not introduced yet) to implement the analyzer through docking API. The most recent code update was two years ago, and there are currently 176 stars.

Https://github.com/google/error-proneerrorane is also produced by Google, and it has been supported to be integrated on shipshape (also supported by checkstyle and findbugs). It is mainly used in the Java compiling system inside Google to find serious code mistakes. It is characterized by static type analysis and inspection, which can find some errors that are not easy to be found during compilation. The advantage is that hooks is in the process of building, and high maturity can ensure integration in CI. Support rapid deployment in Maven and gradle scripts.

Https://github.com/facebook/infinifer is a static analysis language written by Ocaml and produced by Facebook. It supports Java, Object C and C. Java and Android programs can be checked for null pointer exceptions, free disclosure, resource competition conditions and other vulnerabilities. The market reacted well with star number of 8182.

Fireline - http://magic.360.cn/zh/index.html360a free static code analysis tool developed by the qtest team of the Web Platform Department. Mainly for mobile Android products static code analysis. Its most outstanding advantage is the comprehensive detection of resource leakage. At the same time, FireWire cooperated with 360 information security department to launch a series of detection rules for mobile security vulnerabilities. Free to use, and support Android studio plug-in, Jenkins plug-in, gradle deployment and other integration methods.

Https://spotbugs.github.io/spotbugs focuses on the old bug detection tools of Java programs. There are many supporting rules and types.

https://github.com/find-sec-bugs/find-sec-bugs/

Findsecuritybugs focuses more on the discovery of Java Web and Android vulnerabilities, with 125 types of vulnerability judgment and 787 API feature detection.

According to the collected industry vulnerability demo, 30 test cases have been implemented. At present, the architecture of springmvc + FreeMarker + mybatis is supported. It tries to show the principle and scenario of vulnerability as much as possible, including XSS, JSTL, arbitrary file upload and download, file inclusion, SQL injection, spring expression, command execution, deserialization, SSRF, directory traversal, xxE, high-risk function reference Multiple test cases. It is hoped that through such threshold use cases to evaluate the use effect of various open-source code analysis software, to achieve the data support of leak finding and defect filling, and to realize the utility of "fetchism".

Test source address: Na

The test method is to submit the build task through the coverage, and smoothly perform the check after a five minute cycle. The default configuration view item takes 8 min and 30 sec, including 11 sec for sequence waiting, 8 min and 19 sec for checking out, build, compile, analysis and commit. The bottleneck lies in the setting of platform system software and hardware configuration, database performance and compilation parameters. The display effect is as shown in the following figure. The problem detected in the security view is 0, and the problem of 5 null type references is detected in the quality class.

After simple tuning, the step is to remove the misconfigured - Security (only valid for C, C + + projects). – disable FB (will remove the findbugs plug-in), add the – webapp security preview and – webapp security options. After re running, it took 3 min and 56 sec in total, and 7 new security problems were found, such as redirection, file download, information disclosure, etc.

It can be seen that the effect is significant. In addition, it can be compiled and checked against JSP files (JSP is a special java file, and the tag library and template engine can be converted into java files). Generally speaking, there is still a huge space for optimization after familiarity. Other open source software:

The shipshape test step is to use docker to build shipshape and fill in the source directory. Centos7 environment failed to run, https://github.com/google/shipshape/issues/124. However, considering that it is an integrated platform, there are no effective rules that we can use, so it is not in-depth. This is recorded as a legacy issue, which will be followed up.

Errorpone is a simple tool to use. It can be checked through Maven construction phase. No problem has been detected yet. The principle of the tool is to try out the form of hook compiler. When compiling in javac phase, the ability to detect Java server-side projects is limited.

Inf uses inf to analyze the source code, and the result is 0. In principle, there are three reasons for this: infer is used to detect Android and IOS applications, and the vulnerability detection rules will be less in java development scenarios; secondly, infer runs in the capture phase first, which supports the native javac better, and has poor effect on Maven. It is worth mentioning that the intermediate file of infer transformation source code Ocaml data structure. The. Cfg file contains the control flow of each function or method in the code file. . CG contains the call relationship of the function defined in the code file and the call relationship of the external function in the file. . tenv contains the types defined and used in the code file, which is worth our reference. The second step is the analysis function. If it is not associated, it will stop. Then it is not supported at all for J2EE framework, and it is difficult to realize intelligently.

Fireline:

360's tools also failed to find relevant security vulnerabilities, as shown in the following figure: 5 valid code specification issues were reported. The similarities and differences between this tool and infer can be shown in a picture:

This tool is mainly for mobile applications, but the idea of classifying and sorting rules in terms of security rules is worth learning.

Spotbugs found three defects, the effect is not as significant as find security bugs, but it is a tool that supports GUI pattern in the comparison project.

Find sec bugs found six defects and four high-risk code vulnerabilities, with significant effect.

Through a series of trials and experiences, we can gradually sort out the general trend of open source static code scanning software projects: a small amount of support for docker deployment, open API,

Provide gradle, ant and Maven building methods, and provide a small amount of plug-in support integrated in eclipse and idea. Software is not designed to be large and complete for multiple languages at the beginning, but focusing on small and precise rules. In addition, for the project of node.js, there is an open-source tool https://github.com/ajinabrahm/nodejsscan. For Pandit of python, this article will not introduce golint of go.

Analysis and design ideas and concepts

The basic process of static code detection tool is as follows: for some obvious features, regular rules can be used to directly match them, such as hard coded passwords, wrong configuration, etc., and the rules of rasp project can also be used in this respect, and the efficiency of regular rules will be a problem. For the vulnerability of OWASP top 10, by combing the functions that can cause harm in advance, and locating all the places in the code where the harm function occurs, the corresponding source code is resolved to ast (abstract syntax tree) based on Lex (lexical analyzer generator, lexical analysis generator) and yacc (YT another compiler compiler compiler, compiler code generator), Abstract syntax tree), analyze whether the input parameter of the hazard function is controllable to determine whether there is a vulnerability. The bytecode of the operation class is returned to the interpreter for execution, which can be implemented using ASM or javassist technology.

Excellent company practice:

Google

Using a code review system like Gerrit to ensure the basic quality, we have used various commercial software for several years. Error prone is used in Google's Java build system to discover and reduce various serious bugs.

Ali

The message shows that the internal SDL of Ali was implemented earlier, using a software called STC. S has been promoting security coding, and also has a self-developed source scanner. The main reason is that the project cycle is short, the release is fast, there are many projects, and there are few security personnel, so we can only take the automation route as much as possible, but the automation can not be achieved at present, such as vulnerability and code analysis, architecture design security audit.

HUAWEI

Huawei uses the commercial security tool platform and has custom rules, but does not develop its own products.

Product manufacturers in business segments

Gartner Magic Quadrant on application security testing:

Coverity

Synopsys is a coverage manufacturer, providing a variety of products in the field of software and semiconductor. Last year, Synopsys completed its acquisition of black duck (security testing of open source products). The company integrates application security products with seeker Iast and condensomicon, protectode and coverage of digital, quota, and provides Iast, SAST and SCA functions for Synopsys.

Leading edge

Seeker is still one of the most widely used Iast solutions, providing extensive language coverage and good SDLC integration. Synopsys introduced agent only Iast for seeker, without the need for an importer. Some passive test modes provided by Iast are supported.

Secureassist is well suited for the Devops pattern because it provides a powerful integration with the IDE to provide the SAST spell checker early in the development phase. Synopsys uses the coverage engine to introduce support for JavaScript analysis in secureassist.

Synopsys provides a complete set of AST products suitable for various use cases, including various fuzzy functions (input fuzzy, protocol, etc.) provided by defenses.

Synopsys has a strong position in the IOT ast field. It supports various protocols, such as XMPP, mqtt, COAP and AMQP (through defenses).

Fortify

Micro focus is a global provider of AST products and services under the fortify brand. The products have strong influence in North America, Europe and Asia Pacific markets. Fortify provides static code analyzer (SAST), webinspect (DAST and IAST), software security center (its console) and application defender (monitoring and rasp). Fortify provides its ast as a product and a product in the cloud through fortify on demand (FOD). Mobile AST is available through FOD. Fortify's SAST can take advantage of real-time online vulnerability detection via the spell checker in the eclipse IDE, known as the security assistant.

Over the past year, micro focus verify has introduced incremental scanning capabilities for webinspect to continuously test only changes to web applications. Multithreading is introduced into the SAST product to help improve scan time. In addition, the improvement of vulnerability verification by machine learning assisted audit reduces the turnaround time of SAST. Micro focus fortify's ast products should be considered by enterprises looking for comprehensive ast functions, whether as products or services, or both, with enterprise level reporting and integration capabilities.

Leading edge

Fortify is a global brand. In the customer list of extensive ast use cases, especially when multiple testing technologies are required. It is famous for providing innovative products and services. Having one of the most complete SDLC integrations, for example, out of the box integration for popular IDE and CI / CD tools. (better use, known as workbench) fortify's SAST has the most extensive language support, and provides a series of deployment options, making it very suitable for complex test cases. Its webinspect DAST tool customers provide their webinspect Iast for Java and. Net agents for free. Fortify continues to develop innovative automation and machine learning based capabilities to support Devops, such as real-time analysis in the eclipse ide using security assistants. Internal deployment and fortify on demand customers can use machine learning based audit assistants to delete the SAST findings, and the smartfix feature will provide the best location for repair. Fortify has comprehensive enterprise capabilities and integration with major SCA vendors. All FOD SAST customers can enjoy the sonatype assessment without extra charge.

IBM

IBM provides SAST and DAST desktop tools, including IBM security appscan source, IBM security appscan standard and enterprise platform (appscan enterprise). This includes a centralized management console that enables users to import survey results from third-party tools. IBM security application on cloud for SAST and DAST. Iast is provided through the glassbox agent in appscan (appscan Standard Edition, enterprise edition and cloud Edition), free of charge to DAST customers, mobile ast (mast; IBM mobile analyzer) and SCA products (IBM security open source analyzer [OSA]). For SCA, they license vulnerabilities and fix databases from whitesource. IBM also worked with prevoty to develop rasp. IBM has improved intelligent code analysis (ICA) and extended intelligent survey analysis (IFA) to local customers at no additional cost. Both of them improve the speed and accuracy of SAST scanning results. ICA detects APIs in the language and framework and determines the security impact of these APIs to reduce false positives. IBM IFA uses machine learning to significantly reduce the number of overall vulnerabilities and false positives, correlate the results, and propose the minimum number of code changes to fix vulnerabilities. IBM has a considerable customer base and integrates SAST, DAST and IAST into a set of products and services.

Leading edge

IBM has been expanding its capabilities and focusing on the needs of devosecops. This includes extended language support, dividing DAST interface into developer mode and security expert mode, and running faster and lighter scanning to shorten turnaround time. It is a large stable provider of complete ast solutions (SAST, DAST and IAST) and other security products / services with multi regional presence and delivery capabilities. IBM's application security management provides unified risk centric reporting and dashboard capabilities, as well as IBM's security framework and risk assessment, which is the basic framework for managing security risks in business impact applications. IBM is one of the few vendors to allow the import of third-party ast result reporting dashboards, such as manual code review, penetration testing, vulnerability assessment, and competitor ast solutions.

other aspects

In the planning, design, implementation, verification, release and regression stages, the industry pays attention to the participation of source code scanning:

Static application security analysis - find and automatically repair software bugs and quality defects in code;

Software component analysis: find out whether open source code components or third-party components contain security vulnerabilities and license legal issues;

How to build a safety and quality gate in the automatic integration phase? How to ensure the application security before release?

© 2023