Back in the early 2000s, some didn’t think that the future looked promising for Intrusion Detection Systems (IDS) and Intrusion Prevention Systems (IPS). Gartner was among those voices. As reported by CSO, the research and consulting company said that IDS had surpassed its expectations, had begun to decline, and would meet its end by 2005. Gartner said something similar about IPS security, noting that the technology’s functionality was moving into other solutions such as firewalls and antivirus platforms.

More than 20 years later, security technology has obviously evolved, but that doesn’t mean IDS and IPS have disappeared – far from it. I’ll explain the reasons for this in a second. First, we need to begin with some definitions as well as a brief history of IDS and IPS.

An Overview of IDS and IPS

An IDS tool monitors network traffic for anomalous activity such as suspicious events and violations of network policy. If or when it detects these anomalies, an IDS solution can take one of two responses depending on its features. Passive IDS solutions can send an alert to a Security Information and Event Management (SIEM) system, for instance. The SIEM can then correlate that information and analyze whether the alerts point to a larger security incident. Meanwhile, Active IDS protection (also known as IPS, and run inline on the wire) can take defensive action against those anomalies by modifying the access control lists on firewalls to block suspicious networks, or redirect network traffic to a sinkhole.

IDS and IPS both started with an open-source product called Snort in 1998. Cisco bought Snort in 2013 after it acquired Sourcefire, the company responsible for creating Snort. This product still exists, but most players in the security industry have since moved on to Suricata. IDS and IPS have matured a lot since the early days of Snort; Suricata’s engine does more than just those elemental features nowadays. Along those same lines, there’s a Network Security Monitoring (NSM) project called Zeek (formerly known as Bro) that seeks to provide extended functionality and network visibility beyond IDS.

All this to say, Snort, Suricata, and Zeek probably form the foundation of a very high percentage of any of the network detection and response (NDR) products that exist today. It’s the place where anyone creating an IDS/IPS or NSM product would start. That includes the big guys.

Understanding the Staying Power of IDS/IPS

The ongoing value of IDS/IPS rests with how they help security teams to address two categories of network events. These are known bad and unknown bad. Let’s begin with the former.

Known Bad

IDS/IPS tools are traditionally based on signatures. Developers write rules (or signatures) that are based on samples of malicious traffic they’ve researched. Those signatures match patterns of known bad traffic that are specific to vulnerabilities, or problems found in software, and exploits, or attempts to take advantage of vulnerabilities. IDS/IPS help to recognize the pattern of an exploit taking advantage of a vulnerability. In that sense, it helps to detect something we already know.

Security teams have other ways of defending against known bad network events, as well. Take threat intelligence as an example. By definition, threat intelligence covers things for which we already have intelligence—that is, things we’ve already seen before. This intelligence covers bad domains, IPs, Indicators of Compromise (IOCs), and other artifacts.

There’s also DNS filtering (otherwise known as sinkholing). This happens when someone attempts to make a request to a site that’s known bad. To protect the user, their DNS request to the site gets dropped into a hole and remains there unresolved.

All of these ‘known bad’ tools can potentially save a user from visiting a known bad website and potentially suffering a malware or ransomware infection, but there are other benefits, too.

The Value of Known Bad

We’ve been taking care of known bad events for a long time. That’s because it carries various benefits. First, it’s relatively easy to do. We have the tools (like IDS/IPS) that we need to do it.

Second, known bad events tie in directly to known vulnerabilities and exploits. There are lists and libraries out there that tell us what vulnerabilities are out there and how attackers are seeking to abuse them. We can use that knowledge to defend our systems.

Third, taking care of known bad events helps to reduce the noise coming into analysts. It provides a means of removing traffic so that security professionals don’t need to worry about events that don’t concern them. This carries the added benefit of lowering the false positive rate. Indeed, security teams can tune known bad rules and signatures to get their false positive rate down. This allows them to dynamically block known bad traffic using threat intelligence and DNS sinkholing without getting involved directly.

Fourth, organizations can run IDS/IPS on premises through hardware, VM, or a cloud-based deployment. Let me clarify: the machine that’s running IDS/IPS can do all the processing in real-time. This allows the tool to make decisions very quickly, thus opening opportunities for automation.

Finally, by helping to reduce the network noise, IDS/IPS tools help to take away a lot of what a new shiny tool concerned with detecting unknown bad events would need to deal with. That new solution can just concentrate on what’s left. This gives teams a means of blending their security approaches for known bad events and unknown bad events together.

Unknown Bad

Unknown bad events consist of picking out patterns that can’t be detected by signatures. In other words, their purpose is to identify things that have never been seen before.

To detect unknown bad events, security teams need to first work with their solutions to set a baseline for what they think is normal traffic. They can then use AI and machine learning algorithms to try to find patterns that don’t match what’s considered normal.

This is the idea around rooting out zero-day vulnerabilities. This notion is often overused in marketing campaigns, a practice which corrupts our understanding of how these threats relate to our systems. Fundamentally, zero-day vulnerabilities are, by definition, things that quite literally no one knew existed until the day they were discovered. A known bad tool can’t defend against this kind of threat, which is why we need another means of detection. Using AI and other security technologies, we can detect patterns outside of the norm that we otherwise don’t know about.

The Value of Unknown Bad

As discussed above, addressing unknown bad events helps us to defend against zero-day vulnerabilities and exploits. This involves AI and machine learning tools more often than not. Those types of solutions take network traffic and ship it off to cloud-based tools to do the processing using big data and cloud computing. In doing so, those tools must continue to improve their speed of analysis so that security teams can react quickly in the event of a potential security incident.

There are certain challenges associated with detecting unknown bad events, however. Those types of solutions can be noisy and prone to generating false positives, for example. It’s not a surprise why. AI, machine learning, and other tools like them take less of a “I know you’re bad” approach and more of a “Take a look at this” stance. In other words, it’s harder to determine whether to block traffic identified by these tools dynamically. These tools are more for analysts in that sense. They help to amplify those individuals’ actions, though it’s worth pointing out that many AI and machine learning solutions are maturing and getting better.

IDS/IPS Not Going Anywhere

Regardless of their advertised capabilities, most tools these days still integrate both IDS and IPS functionality. There’s a lot of energy and focus on AI and ML in particular, and for good reason, but most products still use IDS/IPS at the heart of their toolbox.

What’s more, no one in the industry is actually considering dropping or foregoing IDS/IPS (or known bad tools more generally) in favor of unknown bad tools. The latter just adds on something more sophisticated to something that already works. As such, these technologies stack on top of each other. They’re complementary as part of a comprehensive security strategy.