May 5, 2010
As an organization that performs a large volume of code reviews and penetration tests, NetSPI is frequently asked which type of application assessment is the best option. Your primary options are a code review or a web application penetration test. Both are recommended and both find many of the vulnerabilities commonly found in web applications as defined by the Open Web Application Security Project (OWASP) Top 10 (http://www.owasp.org). By themselves, neither a code review nor a web application penetration test find all of the vulnerabilities that threaten the application.
Why perform them?
Many regulations either require them or highly recommend them. For example, Payment Card Industry Data Security Standard (PCI-DSS) requires that either a code review or a web application vulnerability security assessment (web application penetration test) is performed annually on any web application that stores, processes or transmits credit cards (PCI Requirement 6.6). In addition, for payment application vendors (i.e. point-of-sale application, etc.) the PCI Payment Application Data Security Standard (PA-DSS) requires a code review and a penetration test targeting the OWASP Top 10.
Picking one over the other
Even though code reviews and web application penetration tests can find most of the same vulnerabilities, they look at the application differently and as a result their findings can differ. Typically both approaches find OWASP Top 10 issues such as SQL Injection, cross-site scripting (XSS), etc. However, the efficiency and effectiveness how each method finds these vulnerabilities can differ. For example, code reviews are better at finding most instances of input validation issues (i.e. XSS or SQL Injection). All of the automated code scanning tools NetSPI uses trace the data within the application from its entry point to its exit point. A web application penetration test can find these instances but it could take days or weeks to prove they exist in the application.
Web application penetration testing
A web application penetration test can uncover vulnerabilities from the outside looking in. These vulnerabilities can be related to configuration or versions. An example might be an older version of Apache with a chunked encoding overflow vulnerability (http://osvdb.org/838). Web application penetration tests can also uncover vulnerabilities that are related to the operation of the application, such as default or easily guessed credentials. Other types of vulnerabilities found by a web application penetration test include:
- Access control (forced browsing, etc.)
- Session hijacking
- Vulnerabilities related to business logic
In addition, web application penetration testing can find these instances easier than a code review. This being said, I am talking about a third-party code review, not a code review done by a person or person familiar with the code or the company’s development processes. Automated source code analysis tools do not find these types of vulnerabilities. Manual testing can be done but could greatly inflate the cost of the code review.
A code review looks at the application from the inside out. Vulnerabilities commonly found in a code review that cannot be easily found in an application penetration test include logging of sensitive data or application backdoors, as they are not exposed to the outside. Other types of vulnerabilities found by a code review include:
- Denial of services caused by not releasing resources
- Buffer overflows
- Missing or poor error handling
- Dangerous functions
- Hardcoded password or keys in the source code
- Code implementation problems
- Missing or poor logging
Look at the statistics
NetSPI has found through our own experience that only a small subset of vulnerabilities found by code reviews and application penetration tests overlap. In NetSPI’s analysis, only 18% of the overall vulnerabilities found were discovered in both the code review and the web application penetration test where both services were performed on the same application (see Code Review Percentages pie chart below). Of the vulnerabilities found in the web application penetration tests, only 14% of the findings are found by both the web application penetration test and a code review (see Pen Test Percentages pie chart below).
The most comprehensive approach to finding security vulnerabilities in web applications is performing both a code review and a web application penetration test. For critical applications, performing only one of these services can result in many vulnerabilities remaining within the application and unacceptable risk to the organization.