I’ve talked to many schools in the market for MDR lately, but haven’t found a good guide on how to go about choosing one. I reached out to a friend who was going through the process and was able to glean some great information. The following is my Q&A with K12TechPro community member Jared. Jared has worked in K12 for 10 years and is one of my go-to colleagues for cybersecurity advice.
*Disclaimer: These are not the opinions of K12TechPro, rather a frank and blunt conversation with Jared.*
Q: What’s an MDR, and why do schools need one?
A: MDR stands for Managed Detection and Response. Many of us don’t have the time or knowledge to analyze all of the traffic, alerts, patterns, IOCs, and events that are happening within our organization; and this solution allows for another set of eyes on your network to monitor and respond to threats.
Q: You’ve had experience selecting an MDR in the past. Can you take us through the process you go through in selecting one?
A: My organization currently uses an MDR that we’ll refer to by code name “Frigid Wolf.” When the renewal quote came in from our preferred partner, it was much higher than the year before. This led me to believe there had to be alternatives. I quickly began googling. I checked reddit threads and found a lot of alternatives, too many. In order to quickly narrow those down, I used a combination of Gartner, Forrester, and my secret weapon when it comes to security offerings, MITRE Engenuity ATT&CK evaluations.
I don’t put too much value in Forrester and Gartner, as I believe those with the most money make it to the top more easily. But, if the solution doesn’t show up on either chart, I know it is too new for me to get involved. John Strand from BlackHills security has additional thoughts about this in which he describes how these are based off of sales pitches and not actual functionality of the tool. This is why I use MITRE Engenuity evaluations to determine what is actually good.
The first year of MITRE evaluations took place in 2022. I really focused on MDRs that did not also have their own EDR. If I ever switch EDRs, I don’t want to be forced to change my MDR. I also worry about something like Sentinel One or Crowdstrike being able to take in third-party data.
Q: OK, so get started with Gartner and Forrester to understand the players and then use MITRE for your real evaluation of the performance. Did those steps help you narrow the search, and where’d you go from there?
A: After reviewing the MITRE I immediately focused on some of the players. I’ll refer to them by codenames “Lucky Cheetah,” “Red Bird,” and “Blue Bird.”
Using the MITRE evaluation is simple.The link in the article will take you to the overview. It shows which techniques and tactics were involved in the attack, the environment the attack occurred in, and the participants. Once you select the participant to review, you will see a large green button at the top that says “Download OilRig Archive.” I encourage you to click that icon and extract the .zip file. It contains great examples of the emails and alerts you would have gotten if you used the product during the incident.
You can see on the left side the 10 steps (tactics) the attackers performed. Once you click on the Tactic, you will see the techniques that occurred during the attack: anything in purple was detected and reported on, anything in black was not reported on. When you click on a technique that was reported on, you will see screenshots of the log and possibly a write-up of what occurred. I have tallied up all the detections in the chart below.
You may notice that the highest scores are those that create EDR solutions. If you read through the reports each one provides, you will see they only rely on their EDR solution. I do not want to use one of these companies because I believe there are gaps in their coverage. Would they be able to collect and decipher my other non-EDR products such as DUO, M365, Cisco Umbrella, ASA firewall, Mimecast, and other syslogs like AD, DHCP, DNS? Would they be able to detect that a user in Asia just signed into my M365 account? Would they be able to detect and show the AD failed login attempts? Because of those reasons, I will be comparing the “Lucky Cheetah,” the “Red Bird,” and the incumbent, the “Frigid Wolf.”
Q: What can you tell me about the incumbent and the qualifications of it that you had to search for in a new product?
A: Codename: “Frigid Wolf”
I have been using “Frigid Wolf’s” services for the last two years. In the beginning it was awesome, as we had nothing in place before it to compare it to. We didn’t consider other options because it was a last-minute thing and we were naive to the alternatives.
In order to replace this MDR, I needed to find a solution that could, at a MINIMUM, do all the same things the “Wolf” could. At its core, the “Wolf” is an MDR allowing 24/7 SOC as a service. We also purchased the Risk Management add-on. We tied in our third-party software such as DUO, S1, ASA syslog, Mimecast, Azure, M365, Cisco Umbrella. We installed the inline sensors to capture all north, south network traffic. We configured the Risk Scanner to look for vulnerabilities. We took a pass on the Managed Awareness offering, as we are using Knowbe4.
The first year was OK. We had many alerts that we altered to increase the threshold before the alert occurred, or canceled the alert completely. We met with our “Concierge Team” once a month to get suggestions on what we should change in our environment.
Q: On the hunt for a new product, tell me about the two big competitors you looked into.
A: Codename: “Red Bird”
I was able to demo this product. The layout and write-ups of the incidents and investigations were done well. You are able to view raw logs and create your own reports. It does have SOAR integrations. It does allow for actions to be taken by the SOC with Azure or SentinelOne. However, the product does not integrate with DUO or Mimecast (it is on the roadmap, though). There are no sensors to put in place and no agent to install. The “Red Bird” gathers information from the EDR you already deployed to your machines (CS,CB,S1). “Red Bird” uses gathered data to perform threat hunting and determine attacker actions. One big negative for me was the lack of a vulnerability option to their solution. “Red Bird” is staffed by a 24/7/365 SOC. “Red Bird” also offers monthly meetings with a concierge team, much like “Frigid Wolf.”
Codename: “Lucky Cheetah”
I demoed and set up a trial for this product. The solution I compared is MTC (Managed Threat Complete). This solution includes “Lucky Cheetah” and had the best raw log search and useability. It provides many built-in reports ready to be added to the dashboard but also allows fully customizable reports. “Lucky Cheetah” is able to ingest logs from everything I need it to. They provide one agent that is used for log collection as well as vulnerability scanning. The product provides a concierge team for monthly meetings as well as a 24/7/365 SOC. “Lucky Cheetah” also allows for SOAR integration with EDR, Mimecast, Firewall, and many of the other tools in my security stack. These can be created by myself or by my concierge team. “Lucky Cheetah” also allows for logic in the SOAR to send a text requesting a yes or no response to continue with the remediation.
Below are some examples of the log searching views.
Q: After reviewing all the products, can you summarize the differences and tell us where you ultimately landed?
A: “Frigid Wolf” comes with an incident response retainer; however, “Lucky Cheetah” comes with unlimited incident response. You can even ship hardware to them and they will assist you (employee wipes hard drive). “Lucky Cheetah’s” documentation of how to integrate third-party tools is so much easier to follow than “Frigid Wolf’s.” The pricing for both “Lucky Cheetah” and “Red Bird” was basically the same, and since “Red Bird” did not offer the vulnerability management piece, “Lucky Cheetah” was a closer 1:1 match with “Frigid Wolf.” The quote for “Lucky Cheetah” was 54% cheaper than “Frigid Wolf.”
To summarize, I paid 54% less to get better of the same features I had with “Frigid Wolf,” plus SOAR, unlimited IR, and access to my raw logs.
A big thanks to Jared for sharing his helpful and blunt thoughts!
Have you been searching for an MDR? What are some of the things you’ve been considering in your search?