As an organization that performs a large volume of code reviews and penetration tests, NetSPI is frequently asked which type of application assessment is the best option. Your primary options are a code review or a web application penetration test. Both are recommended and both find many of the vulnerabilities commonly found in web applications as defined by the Open Web Application Security Project (OWASP) Top 10 (http://www.owasp.org). By themselves, neither a code review nor a web application penetration test find all of the vulnerabilities that threaten the application.
Why perform them?
Many regulations either require them or highly recommend them. For example, Payment Card Industry Data Security Standard (PCI-DSS) requires that either a code review or a web application vulnerability security assessment (web application penetration test) is performed annually on any web application that stores, processes or transmits credit cards (PCI Requirement 6.6). In addition, for payment application vendors (i.e. point-of-sale application, etc.) the PCI Payment Application Data Security Standard (PA-DSS) requires a code review and a penetration test targeting the OWASP Top 10.
Picking one over the other
Even though code reviews and web application penetration tests can find most of the same vulnerabilities, they look at the application differently and as a result their findings can differ. Typically both approaches find OWASP Top 10 issues such as SQL Injection, cross-site scripting (XSS), etc. However, the efficiency and effectiveness how each method finds these vulnerabilities can differ. For example, code reviews are better at finding most instances of input validation issues (i.e. XSS or SQL Injection). All of the automated code scanning tools NetSPI uses trace the data within the application from its entry point to its exit point. A web application penetration test can find these instances but it could take days or weeks to prove they exist in the application.
Web application penetration testing
A web application penetration test can uncover vulnerabilities from the outside looking in. These vulnerabilities can be related to configuration or versions. An example might be an older version of Apache with a chunked encoding overflow vulnerability (http://osvdb.org/838). Web application penetration tests can also uncover vulnerabilities that are related to the operation of the application, such as default or easily guessed credentials. Other types of vulnerabilities found by a web application penetration test include:
Access control (forced browsing, etc.)
Session hijacking
Vulnerabilities related to business logic
In addition, web application penetration testing can find these instances easier than a code review. This being said, I am talking about a third-party code review, not a code review done by a person or person familiar with the code or the company’s development processes. Automated source code analysis tools do not find these types of vulnerabilities. Manual testing can be done but could greatly inflate the cost of the code review.
Code Reviews
A code review looks at the application from the inside out. Vulnerabilities commonly found in a code review that cannot be easily found in an application penetration test include logging of sensitive data or application backdoors, as they are not exposed to the outside. Other types of vulnerabilities found by a code review include:
Denial of services caused by not releasing resources
Buffer overflows
Missing or poor error handling
Dangerous functions
Hardcoded password or keys in the source code
Code implementation problems
Missing or poor logging
Look at the statistics
NetSPI has found through our own experience that only a small subset of vulnerabilities found by code reviews and application penetration tests overlap. In NetSPI’s analysis, only 18% of the overall vulnerabilities found were discovered in both the code review and the web application penetration test where both services were performed on the same application (see Code Review Percentages pie chart below). Of the vulnerabilities found in the web application penetration tests, only 14% of the findings are found by both the web application penetration test and a code review (see Pen Test Percentages pie chart below).
Final Thoughts The most comprehensive approach to finding security vulnerabilities in web applications is performing both a code review and a web application penetration test. For critical applications, performing only one of these services can result in many vulnerabilities remaining within the application and unacceptable risk to the organization.
In recent years web application security has gotten a lot of attention. The advent of easy to use web proxies has brought a lot of attention to SQL injection and cross-site scripting vulnerabilities, and developers have taken note. Thick application security/development, however, is lagging in that respect. You can pierce the veil yourself and witness the unprotected underbelly of thick application security, because I’m about to teach you how to use a useful tool called Echo Mirage. Echo Mirage is a versatile local proxy tool that can be used to intercept and modify TCP payloads for local Windows applications. It allows users to launch a program behind its proxy or hook into an existing process. It also supports OpenSSL and Windows SSL. Using this tool sheds light on a whole slew of bugs and holes concealed by the thick application security illusion.
Keep in mind that this technique could be interpreted as reverse engineering. Depending on the license of the software you are testing, this could stray towards the grey side of legality. For the purposes of this tutorial, I have created my own C# SQL command handler.
Step 2: Open up Echo Mirage, and click File-> Execute. Choose the .exe for your file, and click OK. Click on the green Go arrow, and your application should start. Phonebooks, invoicing, and ERP systems are common examples of applications which hook into a database and could be vulnerable to this sort of attack.
Figure 1: Having selected my target executable, the path is listed in black.
Figure 2: After launching the application, the red text demonstrates that Echo Mirage is intercepting traffic from the target process.
Step 3: Initiate a connection to a remote database; while my slapdash SQL interface has a button labeled “connect,” many applications will be less clear about when a connection to a database is created. When I start the connection, Echo Mirage intercepts all the packets that I’m sending to the database. Note that even though the connection string is available, many recent implementations of SQL will encrypt the password before it goes over the wire.
Figure 3: Connection strings! My favorite!
Step 4: Create a query. It will be automatically intercepted by Echo Mirage, and you can relay whatever malicious queries you want. In another application this step could be running a search, updating a record, or generating a report. When sending your request, one limitation of Echo Mirage becomes apparent: it is unable to change the size of the data sent. What this means for a potential attacker is that sending a larger query allows for more space when injecting. There is little worry of sending a query that is too large; if you have extra space at the end of your injection simply comment the rest out.
Figure 4: This is the query as sent from my interface
Figure 5: Echo Mirage captures the request
Step 5: Now that you have the query captured in Echo Mirage, overwrite some characters to inject. Try not to disrupt the formatting and only overwrite characters that were actually part of the query you sent.
Figure 6: The edited query, prior to sending
Figure 7: The results of the edited query
I hope this demonstration hits home and proves the necessity of input validation and parameterized SQL queries, even in thick client environments. As tools like Echo Mirage mature, this type of attack will only become more common and more dangerous.
I’ve always been a firm believer in incorporating manual testing as part of any security assessment; after all, a human is the best judge of evaluating the contents of application output, and best able to truly understand how an application is supposed to function. But after attending Darren Challey’s (GE) presentation at the 2009 OWASP AppSec conference, I was encouraged that someone actually measured the value of manual testing – and justified my belief!According to Darren, no single application assessment or code review product could find more than about 35% of the total vulnerabilities GE could find with a manual process. That alone should encourage anyone serious about eradicating vulnerabilities within their applications to step it up a notch!I would not want to be the person certifying an application for public consumption with only 30% of security issues fixed!
To understand why manual testing is so critical, let’s break down some of the reasons why assessment tools have limitations. For network scanners, vulnerabilities are largely based upon remote OS and application footprints; accuracy will decrease if that footprint is inaccurate or masqueraded. Application scanners must try to interpret application output; if an application uses custom messaging, what’s the scanner supposed to think? Code review products are never going to be able to accurately interpret code comments, identify customized backdoors, or follow application functionality that might appear orphaned. One must also keep in mind that an assessment product will only report on something if the vendor has written a check or signature for it; think about how many vulnerability signature authors exist compared to the number of hackers identifying new exploits.
Automated testing has a very important role in security assessments: these security tools help us identify a large swath of mainstream issues in an efficient manner. And manual testing can be expensive and time consuming. However, the cost of fixing vulnerabilities after an application or system is in production is also very costly and time consuming.According to the Systems Sciences Institute at IBM, a production or maintenance bug fix costs 100x more than a design bug fix. Furthermore, the cost of a breach increases annually as well.Adding comprehensive manual testing to your assessment criteria does have an ROI, and more importantly, could improve your detection accuracy by 60% or more!
According to Gartner, 75% of the attacks are coming though web applications and not through the network. This means greater emphasis needs to be placed on application security. However, this does not appear to be happening.
Application security vulnerabilities are increasing
For the first half of 2009, Cenzic identified about 3,100 total vulnerabilities, which is an increase of over 10 percent from the second half of 2008. (http://www.cenzic.com). Another revealing piece of data: WhiteHat Security has stated that in 83% of the 1,300 websites they scan have had at least one serious vulnerability (http://www.whitehatsec.com). Of the projects NetSPI has done in the application security area, 83% of these projects also had serious findings (serious vulnerabilities are those of HIGH, CRITICAL, or URGENT severity as defined by PCI DSS naming conventions).
What can happen if you do not fix the problems?
The first real risk is the theft of your data or your customers’ data. If applications are not done right, SQL Injection can allow a person (or persons) access to your database. Think TJX and all of the problems they had. Another risk is to your company’s reputation.Given the right situation, a user could be redirected to a site that is not under your control. It could be a porn site or even a site that looks like yours; it just exists to steal your users’ credentials. Your reputation will take years to repair, and the cost to your company may be insurmountable.
What can you do?
Many of the problems can be fixed by training. These do not have to be external training courses; they could just be brown bag lunches that cover specific secure coding techniques. A good place to start is the OWASP web site (http://www.owasp.org ). This site gives good information on detecting and preventing these vulnerabilities.
Perform code reviews and application vulnerability assessments on a regular basis. Code reviews need to be performed every time the code changes. Application vulnerability assessments need to be done at least annually.
By doing code reviews, and vulnerability assessments, you are helping both your company and your customers.
Integrating security checks and balances with your application development processes is certainly uncharted territory for many security professionals. Why is this so? With the multitude of benefits that custom developed applications bring to an organization, there is also a multitude of risks, namely that sensitive, regulated, and confidential data is being stored, processed, transmitted, and exchanged inside and outside the boundaries of the organization. Why don’t we have a better handle on these risks? I think that we as security professionals have missed an opportunity to embed security in the development process. Let’s begin to understand why this is and how we can go about changing it.
Step back a few years to when your security program was just really getting off the ground. Developed applications were supposed to just work right. Applications either met design specs or they didn’t; wasn’t it that simple? Why did security need to be involved? Back in the day, applications were always quarantined behind the firewall, and business partners only received data via a data exchange solution. Out of scope for security, right?
Now, fast forward from the days of firewalls and anti-virus concerns and take notice: everyone who has any reason to access your data can, from anywhere, anytime. Sure, you’re protecting it with a strong password, maybe even two-factor authentication, a VPN, or rock-solid acceptance and confidentiality agreements. You’re conducting vulnerability scans on the DMZ nightly, right? Would you ever dream of scanning that application nightly? In production? Are you crazy? That might cause an outage. But hold on, and consider what else you may be missing.
Good application security practices require security checks and balances throughout the entire lifecycle, not just a security assessment at the end of the line. These checks and balances include things like secure coding standards, developer training, well-defined security requirements, security code reviews, the use of sanitizing test data, separation of duties, security testing during development checkpoints, security use case testing, and penetration testing. So, if that’s where we need to be, how do we get there? Is the answer Security Software Assurance Programs? They sound like a great concept, prescriptive programs that include everything from A – Z for application security; all you need to do is implement it and watch the needle on the risk meter drop.
Let’s get real. Budgets are tight, development timelines are tight, and security programs are underfunded. Can you realistically launch a massive and invasive security program? Once again, I think we missed an opportunity somewhere between the 1990s and now to ensure that security and application development work in tandem.
So what do you do now? I suggest you think big, but start smart. Pick two or three obvious pain points and start there, e.g., with more testing or better requirements. If there is one thing to get in there sooner than later, start training your development leads (literally, scare the daylights out of them). Keep in mind, you will need to turn this into a full-on program, so begin working towards it, but begin by re-acquainting yourself with your developers and providing them with some high value wins. Help them find security issues in-house before someone else does.
Subscribe