FireEye Stories Blog

Delivering the Detections: MITRE ATT&CK Evaluation Demonstrates FireEye Endpoint Security and Mandiant Managed Defense Detection Leadership

In the 2019 MITRE ATT&CK® assessment, announced on Apr. 21, 2020, FireEye Endpoint Security and Mandiant Managed Defense delivered the highest cumulative detections, and the highest number of technique detections. We also provided visibility with a rich set of raw telemetry data and provided enrichments to our alerts with unparalleled managed detection and response service. In this second iteration of MITRE’s ATT&CK framework evaluation of Endpoint Detection and Response (EDR) solutions, they leveraged a sophisticated adversary emulation of APT29, a Russian-backed adversary.

MITRE, an independent not-for-profit organization, created a globally accessible knowledge base of adversary tactics and techniques. The ATT&CK framework is used as a foundation for the development of specific threat models and methodologies in the private sector, government and broader cyber security community.

FireEye is excited to have participated in MITRE’s public evaluation, strongly believes in the value of transparent assessments, and applauds MITRE’s unbiased test methodology. We are extremely proud to have contributed to the knowledge base with our public blog posts and reports covering APT29: NotSoCozy, TweetBlog, and HAMMERTOSS.

FireEye Has the Highest Cumulative Detections Amongst All Vendor Participants Ensuring the Most Comprehensive Coverage of APT29

MITRE evaluated 57 ATT&CK techniques with 134 test procedures. There are cases where a vendor can have multiple detections for the same test procedure/step (e.g. Telemetry and Technique). Cumulative detections are a count of how many detections a vendor had across all five major MITRE categories (General, Technique, Tactic, MSSP and Telemetry), making sure the results are not skewed for just one detection type.

FireEye has the highest cumulative detections of any of the evaluated solutions and the result is a testament to our unique innovation cycle. Our deep understanding of threats is derived from the most robust threat intelligence capabilities in the world combined with insights gathered from more than 200,000 hours yearly of hands-on client-facing attack investigation and response. This unique combination allows for rapid innovation and development of enhanced capabilities in FireEye Endpoint Security.

FireEye Has the Highest Number of Unique and Cumulative Technique Detections Amongst All Vendor Participants

Technique detections are one of the most important detection categories as they directly map to the ATT&CK framework. These detections provide direct information to an analyst on how an action was performed and helps them to understand—with enriched data—exactly what tactics, techniques and procedures were used by APT29.

FireEye identified the highest number (both unique and cumulative) of technique detections of any solution taking part in the MITRE ATT&CK evaluation. This is because FireEye Endpoint Security has the capability to detect the most sophisticated attacks, providing relevant context, impactful telemetry and the most critical alerts. Developed by the world’s best frontline responders, FireEye Endpoint Security offers unparalleled visibility into the most impactful threats.

FireEye Has the Highest Number of Cumulative Product Detections and Telemetry

Telemetry is a foundational EDR capability in the modern security operations center (SOC). Alerting and correlation with telemetry is crucial for an analyst responding to a threat. The alerting capability FireEye offers, with a rich set of raw endpoint telemetry data, provides the most relevant context and additional information that enables effective alert triage and accelerates response. Having rich telemetry data available for the analysts/responders in addition to detections is fundamental to a complete endpoint solution.

Detection Differentiators: Highlighting the Ability to Create Custom Security Content

FireEye Endpoint Security allows customers to create and upload their own security content, in addition to the default content in Endpoint Security. To showcase this capability, FireEye leveraged custom ATT&CK-specific security content, alongside production content, during the evaluation. FireEye has released the ATT&CK security content to our FireEye Market. These custom rules can be used to augment detection and as examples for customers interested in creating their own rules.

Mandiant Managed Defense: A Force Multiplier

FireEye believes that technology alone cannot solve all security challenges and the best security posture includes technology, intelligence and expertise. Mandiant Managed Defense allowed us to showcase the industry’s best detection and response experts, alongside the largest global cyber threat intelligence capability, which harnesses machine, campaign, adversary and victim intelligence gained on the front lines of the world’s most consequential cyber attacks.

FireEye successfully showcased this force multiplier during the MITRE evaluation. Mandiant Managed Defense is a managed detection and response (MDR) service that combines industry-recognized security expertise, FireEye technology and unparalleled knowledge of attackers to identify threats early and help reduce the consequences of a breach. The Managed Defense rapid response capability delivers quick containment of the impact of the threat and provides detailed reporting and analysis on the investigation. The innate focus on delivering real answers to security challenges and being an extension of a customer’s existing security operations is transformational for any enterprise. To effectively defend against today’s sophisticated attacks, organizations need proactive, advanced threat detection and response services.

MITRE has updated their evaluation criteria to include MSSP as one of their main detection types. FireEye had one of the highest number detections that were enriched with enhanced context using Mandiant Managed Defense. This showcases FireEye’s complete EDR solution, backed by the only leader in threat intelligence, and an MDR service backed by the world’s foremost Incident Response consultants.

FireEye Endpoint Security: Finding the Needle in the Haystack

FireEye Endpoint Security provides the most robust endpoint security solution combining FireEye technology, expertise and intelligence to defend against today’s cyberattacks. FireEye uses four engines in our Endpoint Security solution to prevent, detect and respond to threats, as well as providing extensive investigative and threat hunting capabilities:

  • To prevent common malware, Endpoint Security uses a signature-based endpoint protection platform (EPP) engine.
  • To find threats for which a signature does not yet exist, MalwareGuard uses machine learning seeded with knowledge from the front lines of cyber attacks.
  • Exploit Guard, a behavior-based analytics engine, stops exploits and threats from common attacks such as phishing.
  • Endpoint detection and response capabilities are enabled through a real-time events engine that uses current, frontline intelligence to identify advanced threats.

This defense in depth strategy helps protect enterprises by both preventing and reducing detection time of attacks. Native forensic capabilities and the ability to rapidly search EDR data and operating system artifacts at enterprise scale empower analysts and investigators to efficiently search for compromise, determine the scope of attacks and resolve incidents. With additional modules—such as Logon Tracker (enables the investigation of lateral movement within Windows enterprise environments), Process Guard (prevents credential theft by preventing access to credential data or key material stored within the Windows OS), Enrichment (adds Mandiant Threat Intelligence information to files to help determine when a file is malicious and aid in incident response investigations) and Process Tracker (collects metadata on Windows, Mac and Linux endpoints and streams the data to the Endpoint Security console)—FireEye Endpoint Security continues to innovate and respond to the ever-evolving threat landscape.

MITRE ATT&CK Evaluation Timeline

FireEye confirmed participation in MITRE’s evaluation as part of the cohort in July 2019. The FireEye evaluation commenced on Dec. 12, 2019, and finished on Dec. 15, 2019.

FireEye is in full support of MITRE’s collaborative, open product evaluation process, how it will help inform organizations about the action adversaries take, and most importantly, how those behaviors affect security efficacy. FireEye’s strong showing demonstrates our deep understanding of what it takes to protect our users, and our pedigree of having world-class expertise responding to breaches and providing managed detection services, the best threat intelligence, and best-in-class product capability.

Evaluation Methodology

MITRE’s evaluation is a detailed capability assessment of each solution’s ability to detect and respond to techniques used by APT29. It is important to note that the MITRE assessment does not provide any quantitative scoring of the solutions that were evaluated and does not rank vendors.

To offer context on how FireEye Endpoint Security and Mandiant Managed Defense compared to the other solutions evaluated by MITRE, FireEye used the raw JSON results data that MITRE published for all vendors and provided a cumulative update across various categories with no configuration changes. No scores were weighted or leveraged any preferential qualifiers or modifications. Graphs are pure cumulative totals of various detection categories as shown, and can be validated by going to the full results on the MITRE website.

Using these criteria, FireEye Endpoint Security and Mandiant Managed Defense received a best-in-class showing in all key categories evaluated.

Additional third-party validations such as an Approved Business Product from AV-Comparatives, and certification from Virus Bulletin, showcases FireEye Endpoint Security’s leading position in the market, and our commitment to independent third-party testing. Independent confirmation of Endpoint Security’s effectiveness can be shown through its capturing top place in the NAVWAR Artificial Intelligence Challenge.

View details from MITRE on FireEye Endpoint Security’s results. View details on FireEye performance against APT3 during MITRE’s first round of testing. Additionally, take a self-guided tour of Endpoint Security and request a free 30-day evaluation.

Organizations can validate their own endpoint vendor against APT29 as well as the key threat actors targeting their industry using Mandiant Security Validation (formerly the Verodin Security Instrumentation Platform). Request a demo today.