Penetration Testing – Deception through Vocabulary

For those of you who have followed the NetSPI blog, you will (hopefully) have noticed that we do try to make our posts useful and informative.  We’ve kept the rants to a minimum and the speculation non-sensational.  Many of our posts are technical and focused on detailed descriptions of testing techniques.  Some of our posts are less technical and focused on industry trends and advice. This post is not of the technical nature (I’m the wrong guy) nor is it really about industry trends (maybe a little).  I want to use this post to focus on some industry-specific vocabulary.  While there have been those in the security industry that have knowingly mis-used terminology to deceive clients, it seems that the trend is growing and we’d like to take the time to help those of you who read this blog or stumble over this post really understand what a few related, but very different terms mean. Specifically I want to focus on the term ‘Penetration Testing’ and its derivative services. Please note that I’m writing this for both the people outside the security community that are buying penetration testing or penetration testing tools as well as consultants and technical assessors within our industry.  I think that there are many on both sides that are either ignorant or willfully misusing language. Here’s how NetSPI (and our clients) define the term:

Penetration Test – An assessment of an environment or application (or both) that utilizes a combination of automated tools and manual processes to 1) enumerate vulnerabilities, 2) verify the existence of the vulnerabilities, and 3) safely exploit those vulnerabilities to better understand the risk that those vulnerabilities pose to the environment.


Please note that this is a three-part process.  If you only enumerate vulnerabilities it is NOT a penetration test (this is sometimes called a ‘health check’ or is referred to as a ‘scan’ as it is primarily an automated exercise).  If you only enumerate vulnerabilities and verify that they exist it is NOT a penetration test (this is what NetSPI calls a ‘Vulnerability Assessment’). Note also the phrase ‘a combination of automated tools and manual processes’.  If you are only using automated tools and are not manually testing, verifying, and penetrating, you might be able to call what you are doing a ‘Penetration Test’, but it’s a very, very poor one. There are a lot of information security companies out there right now that provide ‘Penetration Tests’ that stop at enumeration.  There are also a lot of companies out there selling ‘Penetration Tests’ that might verify some or all of the vulnerabilities they enumerate actually exist.  Both of these situations are misleading and we constantly have to educate organizations on what the term ‘Penetration Test’ really means.  It has ‘penetration’ in the name, for goodness sake; if there is no penetration how can it be called a ‘Penetration Test’?

Level of ServiceAppropriate Nomenclature
Vulnerability Enumeration through Automated Scanning / Reporting“Scan”, “Health Check”
Vulnerability Enumeration and Verification Through Automated Scanning and Manual Processes“Vulnerability Assessment”
Vulnerability Enumeration, Verification, and Safe Exploitation through a Combination of Automated Tools and Manual Testing“Penetration Test”

I realize that for many of you (most of you, hopefully) this post is nothing new. If so, I’m certainly sorry for wasting your time, but every time I think we as an industry are past this issue it pops up again. I’ve also discovered that non-security executives often seem to think that a pen test is a pen test is a pen test and while this certainly isn’t the case (there is real skill involved in effective penetration testing, as well as the need for a solid process), what’s really frustrating is that it’s often the situation that what people call a pen test is actually a vulnerability assessment or a scan and that drives me nuts. In any case, please let me know if you have any feedback or thoughts on this topic. This is a big one for us – NetSPI focuses very heavily on penetration testing (as well as vulnerability assessment) and, in my opinion, we are the best in the business. Even if you’re in the industry and want to argue with me on that (btw – you’re wrong), I hope that you are doing your part to help educate clients as to the differences between these terms and the levels of service associated with each. Alex Crittenden


Mobile security is the new hotness

Mobile security is the new hotness.  The conventional wisdom hasn’t yet been established, but many security proponents are gunning for users who jailbreak or root their devices.  Symantec and Good both offer enterprise solutions that include features to manage root privileges on employee devices.  Unfortunately, malware engineers just changed their approach. As background, many approaches to mobile security rely on preventing users from gaining root access.  Root access allows a user ultimate control over the phone, regardless of the inherent protections built into the device’s operating system.  Many users who go about acquiring root access do so in order to harmlessly customize their device.  Some users leverage root privileges to subvert controls on functionality like mobile tethering.  In any case, this process is seen as a risk since a user who roots their phone is capable of granting these enhanced privileges to any application that requests escalation.  If a user inadvertently grants root privileges to a piece of malware, that malware could access any data on the phone, including potentially protected, corporate information. In August, a piece of malware called GingerMaster was found to escalate to root privileges on any device compromised.  From a management perspective, it no longer matters whether or not users in a given environment have rooted handsets.  At this point, a user with a rooted device who installs a malicious app is just as likely to expose sensitive or controlled information as a user without a rooted device. This means there isn’t a technical control that can prevent a given user from installing a malicious app and accidentally compromising anything from their email to their entire corporate environment. Just like with SSL certificates, users will have to learn to differentiate between helpful apps and malicious ones.  Thankfully, attackers are still disguising most of their malware pretty poorly.  The cutting edge malware GingerMaster, for example, was disguised as “Beauty of the Day.”

Discover why security operations teams choose NetSPI.