Let’s talk about application vulnerability assessments, penetration testing, and code reviews. How effective they are depends on a number of factors: the education and experience of the testers, the tools used, the restrictions put on the testers, or even the environment in which the testing is done. This post focuses on the education and experience of the testers.
Consider the well-known recent case of the Heartland breach. Robert O. Carr, Chairman and CEO of Heartland Payment Systems, was quoted as saying the following: “In early 2008 we hired a QSA to perform a penetration test which found nothing. On April 30, 2008, we were deemed PCI-compliant” (http://www.infosecurity-us.com/view/4562/qsa-system-is-broken-says-heartland-ceo/).
I wonder if Heartland Payment Systems queried the QSA company on the background of the pen tester. Yes, the company was QSA-certified, but did the person or persons actually doing the penetration test have the education and experience needed to perform a pen test well? Not everyone does. This also goes for application vulnerability assessments and code review. Just because you hire a company that sells itself as having experts on staff does not always mean you get the top dog or even the middle dog. You might be getting a puppy. If the company performing the testing uses a team approach, the team’s collective knowledge might be as good as or better than that of the top dog.
Find out who will be performing your tests and get their resumes, or at least ask them about their background. What kind of training and experience do they have in this area? Are they right out of school or do they have at least a couple of years of experience? Does the firm employ a team of specialists? Is their work process mature and well defined?
These are not hard questions to ask or answer. Making this small effort could make a big difference in the effectiveness of your application security assessments, and your organization’s overall information security.
Subscribe