“Anything worth doing is worth overdoing.”
If you led Mick’s life, you’d be hard pressed to argue with him. Lately, the Mick manifesto has become all too true in cyber security.
Many years ago, an interesting—and dangerous—trend began: security teams started measuring the effectiveness of a security technology by counting the number of alerts generated. The rationale? More alerts = better security. CISOs and their teams were asked for more data to support compliance and gain insight into an ever-changing threat landscape. Vendors answered the call with more alerts. Lots of them. We all over did it.
In my experience, an overwhelming majority of alerts are false positives in most organizations that haven’t set about fixing that situation. Why is that a problem? Too many false positives:
- Cause ‘alert fatigue,’ which leads to the chance to miss something critical – especially since it also means more false positives and less faith in alert fidelity. It’s the classic ‘boy who cried wolf’ dilemma. More alerts also means less time to look at each alert closely for what’s truly important and what’s useless. More alerts are only helpful when those alerts are accurate, and when they don’t cause you to spend time researching one alert (more likely than not a false positive), only to miss the one you really needed to see.
- Mean more expense in the form of missed breaches, which cost plenty to respond to. Alert fatigue also comes at the expense of organizational maturity. It’s like taking one step forward, but two steps back. Every time you take a step to improve (like getting some quality and high-fidelity alerts), you’re dragged backward fighting the deluge of false positives.
- Require more people, and sadly, there aren’t enough of them to go around. There is a crucial shortage of security personnel with the expertise to separate the legitimate from the not-so-important alerts, and to do the detailed analysis demanded by a true positive -- what happened, why, who did it, and when. Our industry doesn’t have enough automation in place to do it without manual oversight. A 2014 report by the IDC showed that 37% of respondents worldwide face more than 10,000 alerts per month, or the equivalent of 14 alerts per hour. It’s a nearly constant occurrence. Since the shortage of security personnel isn’t likely to turn around any time soon, the only other choice is to lower the volume of alerts.
Bottom line: Detecting attacks is important. But detecting attacks while generating a large number of false positives is about as effective as not detecting attacks at all. It merely leaves customers to find the needle in the haystack. This signal-to-noise ratio is simply too low in most places. Remember the old analog TV? When the signal was too weak, all we saw was snow. How can you find an intrusion that way?
How to Process Alerts Correctly: The Alert Processing Algorithm
The time has come to measure security technology differently. As security geeks, applying an algorithm to the process is not just natural, but necessary. What would the algorithm look like?
Effective detection = Detection rate – false positives
Detection rate = % of successful attacks detected by a technology
False positives = % false positives created by the technology (because every false positive investigated takes time away from the security team, which is time they would have otherwise used in investigating a true attack)
Changing the Equation
It’s time to move the conversation away from security efficacy to effective security efficacy. Alerts are the only way to determine what steps to take next. You need the alerts – you just need them to be accurate. In my mind, this equals operational effectiveness. Before you invest in technology, you want to know how effective it’s going to be in your environment to determine the total cost of ownership.
This equation can go a long way in helping you assess alerts and judge the tools that produce them. Because at the end of the day it’s not just about how many alerts you received, it’s about how you addressed them. And if the red herrings get in the way and distract you from the one alert you needed to actually respond to, it’s game over.