Heartbleed: A great time to think about incident response

Heartbleed is the nickname of a dangerous OpenSSL vulnerability that was just announced. A security update was already available before the announcement, and this is definitely a vulnerability where quickly patching makes a big difference. A fast response matters here because malware wasn't in the wild yet, so many sites likely can prevent any negative consequences with quick action.The necessity for rapid response to vulnerabilities illustrates why you should have an incident response procedure in place. An incident response procedure allows for a measured, planned response to a security incident like this one. In this blog post, we'll walk you through the basics of putting together an incident response plan, mostly based on NIST's incident response process.

What is a security incident and what should I do to plan for it?

A security incident is any "violation or imminent threat of violation of computer security policies, acceptable use policies, or standard security practices." This might include attacks from the outside or inappropriate use internally. Every organization should have a definition of what types of events they consider to be security incidents and how they respond to and report different kinds of incidents. Heartbleed should be considered an incident because it's an imminent threat, although not necessarily a violation yet.If you have an incident response procedure in place, you probably already know if you are affected by Heartbleed and have possibly addressed the incident. Without a procedure in place, you might just be reading news reports and wondering if you should care. The difference in these two responses will have a huge impact on whether sensitive information gets leaked and how long it will take to address Heartbleed.More generally, without an incident response procedure, administrators make preventable mistakes in handling or reporting an incident. The development of an incident response procedure is a security best practice according to both SANS' top 20 list of recommended security controls, and NIST 800-53, which recommends incident response (IR-4) for even "low impact" security environments.Here's the basic approach: preparation, detection and analysis, containment, eradication, recovery, and recording lessons learned. SANS suggests that there is a set of quick wins that will help immensely: develop a set of written procedures, including personnel roles; who should report incidents; when they should report them; what information they should provide; and how users are to be trained. Communication with the media can be crucial for high-profile incidents.

• Preparation: Establish the incident response capability in the first place, open the lines of communication to the relevant people inside and outside your organization, and have the correct tools in place, e.g., forensic hardware and software. Most importantly, know who is accountable for watching for incidents and responding to them. With Heartbleed, many administrators were able to patch it before most people had even heard about it.
• Detection and analysis: This step involves discovering the type of incident (in this case, a serious vulnerability in critical software). Are you vulnerable? Do you operate an OpenSSL server? An Intrusion Detection System can often help, but in this case it's all about monitoring vulnerability announcements and knowing your infrastructure. Logging is also important for analysis, so maintaining good logs is important. This step includes documenting the incident as you analyze it, and notifying the appropriate parties. Notification is extremely important since other organizations that you collaborate with might also need to handle the incident in their environments. For instance, if your users' passwords or clients' private data were disclosed, what steps should they take?
• Containment, eradication, and recovery: Containment is about stopping the spread of the incident, whether through privilege escalation or the spread of a virus. For Heartbleed, containment is everything. The longer your systems remain vulnerable, the more likely it is that someone will get around to extracting sensitive information from them. This is a case where, after patching the server, you should go back to the "analysis" step. If an attacker did get into the system, what might they have gotten? If they extracted the server's private key, what could they do with it? Is there any evidence that someone used privileges from this system to get into another system? This step often includes forensics, i.e., gathering information without damaging evidence. Eradication and recovery sometimes involve restoring systems from a known, good backup.
• Post-incident activity: This step involves analyzing lessons learned and cause analysis. Was your team's response effective? Could it have been better? What new processes should be put in place? Retaining evidence for potential legal activity is also important. Recording the process you followed also helps when your users, clients, or the media have questions later about how you handled the incident.

Benefits of having an incident response plan

Again, the detailed documentation from NIST helps to clarify why incident response is important: quicker containment and recovery from attacks, better preparation for future attacks, and dealing with legal issues that can arise. Besides the positive benefits, it can be required by law in some circumstances.

• It helps to make sure the appropriate steps are taken - e.g., patching quickly, making sure that the attacker doesn't gain access to other parts of the system, making sure critical servers don't get taken offline. Without a plan in place, administrators might fail to contain the damage, fail to collect forensic information, and even fail to stop the attack.
• It helps to recover quickly so that disruption and loss is minimized. For Internet retailers, every hour of downtime costs money. For some organizations, every hour lost can damage their mission.
• It helps in learning; incidents should be examined and procedures modified so that the handling of future incidents is improved.
• It helps to deal with the legal issues that arise; it provides steps for collecting appropriate information before the logs are destroyed.

Team considerations

People are everything, and each organization is different. Think through these questions for your organization: How centralized should your incident response team be? How should the team communicate with other parties? Should it be made up of employees or include external parties? Is 24/7 response necessary? NIST also emphasizes providing the team with learning and growth opportunity through training.

Conclusion

Heartbleed is a very serious security incident, and it's likely that your team needs to handle it immediately. Once the dust has settled, take some time to put a security incident response plan in place. It's a stitch in time that will save nine. Contact us if you'd like some help.

About Galois

Galois' mission is to ensure trustworthiness in critical systems. Security incident response helps address today's problem, risk management helps address tomorrow's problems. Our goal is that some day, critical systems can be built with powerful tools to completely eliminate the root causes of many vulnerabilities."