July 30, 2012
Many organizations have Incident Response plans. They go through the testing and send people through training but when that incident happens and the alarms klaxons begin sounding up and down the hallways the response isn’t what the organization expected. This strikes a discordant tone since all audits (be they HIPAA, PCI, etc.) always come back clean as they pertain to Incident Response processes. I’d like to take a brief moment to point of few pitfalls from “on paper” to application, but I’ll leave fixing them to you.
Training is a serious investment for both the organization and the individual. Unless you employ a full-time Incident Response Team this level of training is akin to insurance. Until you need it, it doesn’t seem to really be needed, yet this doesn’t decrease the importance to your organization. Think of them as similar to a volunteer fire department: training is needed even though the volunteers have day jobs. And continued training is critical. Sending people to quality training versus having them read a book from Amazon is like sending that volunteer fire fighter to CPR class versus watching a YouTube video.
Tabletop testing is fine at first but you should strive to get past the tabletop as quickly as possible since this doesn’t simulate the climate of an actual incident. You want the individuals that went to training to use those tools they learned on, assuming they aren’t using them daily. When using just table top exercises you won’t know if everyone is really cut out for Incident Response. Most individuals will agree to participate, but when the rubber meets the road at 3am and the pressure is on to properly contain the incident, are they still willing and able to perform? It’s difficult to admit, but not everyone is cut out for Incident Response. I’m not saying you should get a full simulated environment, but the individuals that went to training should be testing those tools and their skills. Attaching a test scenario to a disaster recovery test is often the most effective and easiest way to minimize additional downtime to production systems. Some tests can be performed that won’t have any impact on the networked environment such as chain-of-custody process testing.
While this often happens, it sometimes is performed more as an accusation arena with pointed fingers and tempers flaring. If needed, have a moderator but keep to the task at hand – determining what happened, what was the full impact, what will prevent the event from happening again, are other systems vulnerable? Write up an agenda of questions and keep people on task.
But what’s really important is to follow up on the remediation work. One great way may be to plug any remediation efforts into your Risk Management program, the only caveat here being that you should keep everyone at the Lessons Learned event informed. If the Incident Response team is not involved in the remediation efforts (even if just informed) then they may not be aware of certain configuration changes that may be relevant for future Incident Responses.
It’s important that you practice the Lessons Learned process by performing them against your tests as well.
As you consider these suggestions, ask yourself: if that event were to occur today, how ready will you really be?