* [Blog](https://www.paloaltonetworks.com/blog) * [Security Operations](https://www.paloaltonetworks.com/blog/security-operations/) * [Must-Read Articles](https://www.paloaltonetworks.com/blog/security-operations/category/must-read-articles/) * MITRE ATT\&CK 2024: R... # MITRE ATT\&CK 2024: Raising the Bar for Security Testing [](https://www.facebook.com/sharer/sharer.php?u=https%3A%2F%2Fwww.paloaltonetworks.com%2Fblog%2Fsecurity-operations%2Fmitre-attck-2024-raising-the-bar-for-security-testing%2F) [](https://twitter.com/share?text=MITRE+ATT%26%23038%3BCK+2024%3A+Raising+the+Bar+for+Security+Testing&url=https%3A%2F%2Fwww.paloaltonetworks.com%2Fblog%2Fsecurity-operations%2Fmitre-attck-2024-raising-the-bar-for-security-testing%2F) [](https://www.linkedin.com/shareArticle?mini=true&url=https%3A%2F%2Fwww.paloaltonetworks.com%2Fblog%2Fsecurity-operations%2Fmitre-attck-2024-raising-the-bar-for-security-testing%2F&title=MITRE+ATT%26%23038%3BCK+2024%3A+Raising+the+Bar+for+Security+Testing&summary=&source=) [](https://www.paloaltonetworks.com//www.reddit.com/submit?url=https://www.paloaltonetworks.com/blog/security-operations/mitre-attck-2024-raising-the-bar-for-security-testing/&ts=markdown) \[\](mailto:?subject=MITRE ATT\&CK 2024: Raising the Bar for Security Testing) Link copied By [Peter Havens](https://www.paloaltonetworks.com/blog/author/peter-havens/?ts=markdown "Posts by Peter Havens") Dec 10, 2024 6 minutes [Must-Read Articles](https://www.paloaltonetworks.com/blog/security-operations/category/must-read-articles/?ts=markdown) [News and Events](https://www.paloaltonetworks.com/blog/security-operations/category/news-and-events/?ts=markdown) [Cortex MITRE](https://www.paloaltonetworks.com/blog/tag/cortex-mitre/?ts=markdown) [Cortex XDR MITRE](https://www.paloaltonetworks.com/blog/tag/cortex-xdr-mitre/?ts=markdown) [endpoint protection](https://www.paloaltonetworks.com/blog/tag/endpoint-protection/?ts=markdown) [endpoint security](https://www.paloaltonetworks.com/blog/tag/endpoint-security/?ts=markdown) [MITRE ATT\&CK](https://www.paloaltonetworks.com/blog/tag/mitre-attck/?ts=markdown) [MITRE ATT\&CK Evaluations](https://www.paloaltonetworks.com/blog/tag/mitre-attck-evaluations/?ts=markdown) [MITRE Engenuity](https://www.paloaltonetworks.com/blog/tag/mitre-engenuity/?ts=markdown) [MITRE Engenuity ATT\&CK Evaluations](https://www.paloaltonetworks.com/blog/tag/mitre-engenuity-attck-evaluations/?ts=markdown) [MITRE evaluations](https://www.paloaltonetworks.com/blog/tag/mitre-evaluations/?ts=markdown) [MITRE testing](https://www.paloaltonetworks.com/blog/tag/mitre-testing/?ts=markdown) [Palo Alto Networks MITRE](https://www.paloaltonetworks.com/blog/tag/palo-alto-networks-mitre/?ts=markdown) Since 2018, MITRE has conducted its annual ATT\&CK evaluations to set the industry standard for measuring the threat detection and prevention capabilities of vendors in the endpoint security market. The experts at MITRE study some of the most sophisticated real-world threat groups, analyze their methods, and develop custom tools that replicate the techniques and tactics these adversaries use to extort hundreds of millions of dollars from businesses annually. This year's evaluations saw big changes that more closely simulate today's changing threat landscape. These changes brought a much higher degree of rigor to the evaluation, making the evaluation results much more impactful. Let's explore how the MITRE evaluation works and how this year was different. ## What's different this year? Participants in MITRE's 2024 evaluations faced new evaluation challenges and more operating system platforms to defend. Each of these changes resulted in a more accurate portrayal of real-world effectiveness. ### False positives In this year's evaluation, vendors faced the challenge of *not* alerting on 20 false positives in the detection phase and 30 false positives in the protection phase. These false positive signals replicated normal business activity that should not have been reported or prevented as a threat. Anyone who has worked in or adjacent to a SOC knows the importance of assessing false positives. Tools that don't create unnecessary alerts on benign activity make life infinitely easier for analysts, allowing them to focus response efforts on what truly matters and prioritize those incidents most effectively. False positives at the prevention stage have the most significant impact, potentially disrupting business-critical processes running on the endpoint. Measuring false positives also guards against spamming tactics from participants --- that is, ratcheting up sensitivity to catch malicious behavior in an evaluation but in practice creating too much noise to be useful to a real analyst. ### Continuous evaluation model This year, MITRE's evaluation ran over several days without breaks, rather than in the neatly defined stages of years past. By adopting this model, MITRE aims to assess how security products perform under sustained pressure, similar to the relentless nature of modern cyber threats. A continuous evaluation model tests the product's ability to correlate events over time. In addition to mounting an immediate response to individual attack techniques, the product must adapt to the evolving threat and create the most complete picture possible of a dynamic, ongoing attack campaign. ### Expanded scope MITRE challenged vendors with a more diverse array of attack types and adversary techniques in this year's evaluation, with attacks targeting Windows, Linux, and MacOS operating systems. Such a broadened scope assesses the participants' abilities to stand up to the speed, sophistication, and scale of modern attacks and the most well-resourced threat actors. ### Inclusion of cloud environments This year's evaluations included more cloud-based attack scenarios, reflecting the growing importance of cloud security and the unique challenges posed by cloud environments. Considering today's hybrid and multi-cloud landscapes, this change is vital for organizations to better understand how these solutions perform in protecting their cloud assets and data --- especially given the focus on out-of-the-box performance. ## The MITRE Evaluation Structure To test the performance of participating tool vendors, [MITRE emulated two types of adversaries](https://attackevals.mitre-engenuity.org/enterprise/er6/). * Ransomware-as-a-Service attacks against Windows and Linux systems. These emulations showcased common features across high-profile ransomware campaigns, like abusing legitimate tools, encrypting data, and disabling critical services or processes. * Attack patterns demonstrated by the Democratic People's Republic of North Korea (DPRK) against macOS. These emulations mimicked multi-staged and modular malware in operations involving elevated privileges and credential targeting. Why these two emulations? Ransomware remains one of the most popular and quickly evolving attacks across the globe, thanks to advanced technology and ransomware-as-a-service models. As for the DPRK, North Korea is one of the most dynamic and sophisticated adversaries out there, regularly targeting high-value systems and organizations. ### Phase 1: Detection [The detection phase of the evaluation](https://attackevals.mitre-engenuity.org/enterprise/er6/detection-categories) assesses how well the solution autonomously identified malicious and suspicious events, met the detection criteria for the tactic, and detailed the technique with which that action was performed. Performance categories are determined by the level of detail provided. * General detection identifies the malicious event and answers the "who, what, when, and where" of that event. * Tactic-level detection identifies the four Ws, but also addresses the fifth W --- "why" the adversary might perform this tactic. Tactic-grade detections link a description at the ATT\&CK Tactic level with the behavior under test., e.g. spear phishing. * Technique- level detection, the highest-performing category, details the five Ws and explicitly links a description with the ATT\&CK (Sub-) technique level with the behavior under test, e.g., spear phishing via attachments. For example, a **general detection** would flag "powershell.exe Invoke-Mimikatz\*" as a suspicious activity. A**tactic-level detection** would additionally relate the activity to credential access. A**technique-level detection** would go even further, specifying how the behavior is related to what occurred, such as credentials stored in the process memory of the Local Security Authority Subsystem Service (LSASS) for OS credential dumping. The evaluations also note "Detection modifiers." Vendors are allowed a chance to adjust their performance, testing whether they can detect an attack they first missed, with some additional manual intervention. In the real world, security teams can't try again on the same attack step, so these types of detections should be used for informational purposes only, not to truly assess the effectiveness of the solution. * A **delayed detection** modifier means an autonomously generated event required manual human augmentation to meet the documented Detection Criteria. * A **configuration change** modifier means the vendor's solution was changed while the evaluation was still in progress. Often, this modifier shows that additional data can be collected and/or processed with new detection content. Detections with a configuration change modifier are identified during an optional 4th day of testing when vendors can redo steps in the evaluation in hopes of achieving a better detection result. ### Phase 2: Protection The Protection phase evaluates whether the tool blocked the malicious behavior. The performance categories for the Protection phase are binary: * The None category indicates the solution did not block the behavior under test. * The Blocked category indicates the solution successfully blocked the behavior under test. The Protection phase also assesses how far the malicious activity progressed before the tool stopped the attack. ## MITRE: A barometer for the cybersecurity world MITRE stands as a thorough, modern evaluation of whether a vendor is keeping up with leading-edge techniques that adversaries deploy against enterprise businesses. We at Palo Alto are grateful to them for helping us discern how we stack up against the industry. For decision-makers choosing products, MITRE provides a valuable scorecard to guide their search. Join us for an in-depth look at the just-released MITRE ATT\&CK Round 6 evaluations and learn how Palo Alto Networks excels in stopping advanced cyberattacks. Hear from experts on key changes, real-world adversary testing, and how vendors performed in this challenging evaluation. [RSVP now!](https://www.linkedin.com/events/7270951481514573825/comments/) *** ** * ** *** ## Related Blogs ### [Announcement](https://www.paloaltonetworks.com/blog/category/announcement/?ts=markdown), [Must-Read Articles](https://www.paloaltonetworks.com/blog/security-operations/category/must-read-articles/?ts=markdown), [News and Events](https://www.paloaltonetworks.com/blog/security-operations/category/news-and-events/?ts=markdown), [Product Features](https://www.paloaltonetworks.com/blog/security-operations/category/product-features/?ts=markdown), [Products and Services](https://www.paloaltonetworks.com/blog/category/products-and-services/?ts=markdown) [#### Forrester Names Palo Alto Networks a Leader in XDR](https://www.paloaltonetworks.com/blog/2024/06/forrester-names-palo-alto-networks-a-leader-in-xdr/) ### [Must-Read Articles](https://www.paloaltonetworks.com/blog/security-operations/category/must-read-articles/?ts=markdown), [News and Events](https://www.paloaltonetworks.com/blog/security-operations/category/news-and-events/?ts=markdown) [#### Cortex XDR and 2021/2022 Forrester Wave Results](https://www.paloaltonetworks.com/blog/security-operations/cortex-xdr-and-2021-2022-forrester-wave-results/) ### [AI and Cybersecurity](https://www.paloaltonetworks.com/blog/security-operations/category/ai-and-cybersecurity/?ts=markdown), [Must-Read Articles](https://www.paloaltonetworks.com/blog/security-operations/category/must-read-articles/?ts=markdown), [News and Events](https://www.paloaltonetworks.com/blog/security-operations/category/news-and-events/?ts=markdown), [Product Features](https://www.paloaltonetworks.com/blog/security-operations/category/product-features/?ts=markdown), [Use-Cases](https://www.paloaltonetworks.com/blog/security-operations/category/use-cases/?ts=markdown) [#### From Silos to Synergy: How Cortex XDL Transforms XDR to Elevate Threat Detection](https://www.paloaltonetworks.com/blog/security-operations/from-silos-to-synergy-how-cortex-xdl-transforms-xdr-to-elevate-threat-detection/) ### [AI and Cybersecurity](https://www.paloaltonetworks.com/blog/security-operations/category/ai-and-cybersecurity/?ts=markdown), [Must-Read Articles](https://www.paloaltonetworks.com/blog/security-operations/category/must-read-articles/?ts=markdown), [News and Events](https://www.paloaltonetworks.com/blog/security-operations/category/news-and-events/?ts=markdown) [#### Cortex XDR is the Only Endpoint Security Market Leader to Achieve 99% in Both Threat Prevention and Response in AVC EPR](https://www.paloaltonetworks.com/blog/security-operations/cortex-xdr-is-the-only-endpoint-security-market-leader-to-achieve-99-in-both-threat-prevention-and-response-in-avc-epr/) ### [News and Events](https://www.paloaltonetworks.com/blog/security-operations/category/news-and-events/?ts=markdown), [Uncategorized](https://www.paloaltonetworks.com/blog/category/uncategorized/?ts=markdown) [#### Palo Alto Networks and MITRE ATT\&CK® Evaluations: Enterprise 2025](https://www.paloaltonetworks.com/blog/security-operations/palo-alto-networks-and-mitre-attck-evaluations-enterprise-2025/) ### [Announcement](https://www.paloaltonetworks.com/blog/category/announcement/?ts=markdown), [Must-Read Articles](https://www.paloaltonetworks.com/blog/security-operations/category/must-read-articles/?ts=markdown), [News and Events](https://www.paloaltonetworks.com/blog/security-operations/category/news-and-events/?ts=markdown) [#### Cortex XDR Named 2025 Gartner Customers' Choice for Endpoint Security](https://www.paloaltonetworks.com/blog/2025/05/cortex-xdr-named-gartner-customers-choice-endpoint-security/) ### Subscribe to Security Operations Blogs! Sign up to receive must-read articles, Playbooks of the Week, new feature announcements, and more. ![spinner](https://www.paloaltonetworks.com/blog/wp-content/themes/panwblog2023/dist/images/ajax-loader.gif) Sign up Please enter a valid email. By submitting this form, you agree to our [Terms of Use](https://www.paloaltonetworks.com/legal-notices/terms-of-use?ts=markdown) and acknowledge our [Privacy Statement](https://www.paloaltonetworks.com/legal-notices/privacy?ts=markdown). Please look for a confirmation email from us. If you don't receive it in the next 10 minutes, please check your spam folder. This site is protected by reCAPTCHA and the Google [Privacy Policy](https://policies.google.com/privacy) and [Terms of Service](https://policies.google.com/terms) apply. {#footer} {#footer} ## Products and Services * [AI-Powered Network Security Platform](https://www.paloaltonetworks.com/network-security?ts=markdown) * [Secure AI by Design](https://www.paloaltonetworks.com/precision-ai-security/secure-ai-by-design?ts=markdown) * [Prisma AIRS](https://www.paloaltonetworks.com/prisma/prisma-ai-runtime-security?ts=markdown) * [AI Access Security](https://www.paloaltonetworks.com/sase/ai-access-security?ts=markdown) * [Cloud Delivered Security Services](https://www.paloaltonetworks.com/network-security/security-subscriptions?ts=markdown) * [Advanced Threat Prevention](https://www.paloaltonetworks.com/network-security/advanced-threat-prevention?ts=markdown) * [Advanced URL Filtering](https://www.paloaltonetworks.com/network-security/advanced-url-filtering?ts=markdown) * [Advanced WildFire](https://www.paloaltonetworks.com/network-security/advanced-wildfire?ts=markdown) * [Advanced DNS Security](https://www.paloaltonetworks.com/network-security/advanced-dns-security?ts=markdown) * [Enterprise Data Loss Prevention](https://www.paloaltonetworks.com/sase/enterprise-data-loss-prevention?ts=markdown) * [Enterprise IoT Security](https://www.paloaltonetworks.com/network-security/enterprise-device-security?ts=markdown) * [Medical IoT Security](https://www.paloaltonetworks.com/network-security/medical-device-security?ts=markdown) * [Industrial OT Security](https://www.paloaltonetworks.com/network-security/medical-device-security?ts=markdown) * [SaaS Security](https://www.paloaltonetworks.com/sase/saas-security?ts=markdown) * [Next-Generation Firewalls](https://www.paloaltonetworks.com/network-security/next-generation-firewall?ts=markdown) * [Hardware Firewalls](https://www.paloaltonetworks.com/network-security/hardware-firewall-innovations?ts=markdown) * [Software Firewalls](https://www.paloaltonetworks.com/network-security/software-firewalls?ts=markdown) * [Strata Cloud Manager](https://www.paloaltonetworks.com/network-security/strata-cloud-manager?ts=markdown) * [SD-WAN for NGFW](https://www.paloaltonetworks.com/network-security/sd-wan-subscription?ts=markdown) * [PAN-OS](https://www.paloaltonetworks.com/network-security/pan-os?ts=markdown) * [Panorama](https://www.paloaltonetworks.com/network-security/panorama?ts=markdown) * [Secure Access Service Edge](https://www.paloaltonetworks.com/sase?ts=markdown) * [Prisma SASE](https://www.paloaltonetworks.com/sase?ts=markdown) * [Application Acceleration](https://www.paloaltonetworks.com/sase/app-acceleration?ts=markdown) * [Autonomous Digital Experience Management](https://www.paloaltonetworks.com/sase/adem?ts=markdown) * [Enterprise DLP](https://www.paloaltonetworks.com/sase/enterprise-data-loss-prevention?ts=markdown) * [Prisma Access](https://www.paloaltonetworks.com/sase/access?ts=markdown) * [Prisma Browser](https://www.paloaltonetworks.com/sase/prisma-browser?ts=markdown) * [Prisma SD-WAN](https://www.paloaltonetworks.com/sase/sd-wan?ts=markdown) * [Remote Browser Isolation](https://www.paloaltonetworks.com/sase/remote-browser-isolation?ts=markdown) * [SaaS Security](https://www.paloaltonetworks.com/sase/saas-security?ts=markdown) * [AI-Driven Security Operations Platform](https://www.paloaltonetworks.com/cortex?ts=markdown) * [Cloud Security](https://www.paloaltonetworks.com/cortex/cloud?ts=markdown) * [Cortex Cloud](https://www.paloaltonetworks.com/cortex/cloud?ts=markdown) * [Application Security](https://www.paloaltonetworks.com/cortex/cloud/application-security?ts=markdown) * [Cloud Posture Security](https://www.paloaltonetworks.com/cortex/cloud/cloud-posture-security?ts=markdown) * [Cloud Runtime Security](https://www.paloaltonetworks.com/cortex/cloud/runtime-security?ts=markdown) * [Prisma Cloud](https://www.paloaltonetworks.com/prisma/cloud?ts=markdown) * [AI-Driven SOC](https://www.paloaltonetworks.com/cortex?ts=markdown) * [Cortex XSIAM](https://www.paloaltonetworks.com/cortex/cortex-xsiam?ts=markdown) * [Cortex XDR](https://www.paloaltonetworks.com/cortex/cortex-xdr?ts=markdown) * [Cortex XSOAR](https://www.paloaltonetworks.com/cortex/cortex-xsoar?ts=markdown) * [Cortex Xpanse](https://www.paloaltonetworks.com/cortex/cortex-xpanse?ts=markdown) * [Unit 42 Managed Detection \& Response](https://www.paloaltonetworks.com/cortex/managed-detection-and-response?ts=markdown) * [Managed XSIAM](https://www.paloaltonetworks.com/cortex/managed-xsiam?ts=markdown) * [Threat Intel and Incident Response Services](https://www.paloaltonetworks.com/unit42?ts=markdown) * [Proactive Assessments](https://www.paloaltonetworks.com/unit42/assess?ts=markdown) * [Incident Response](https://www.paloaltonetworks.com/unit42/respond?ts=markdown) * [Transform Your Security Strategy](https://www.paloaltonetworks.com/unit42/transform?ts=markdown) * [Discover Threat Intelligence](https://www.paloaltonetworks.com/unit42/threat-intelligence-partners?ts=markdown) ## Company * [About Us](https://www.paloaltonetworks.com/about-us?ts=markdown) * [Careers](https://jobs.paloaltonetworks.com/en/) * [Contact Us](https://www.paloaltonetworks.com/company/contact-sales?ts=markdown) * [Corporate Responsibility](https://www.paloaltonetworks.com/about-us/corporate-responsibility?ts=markdown) * [Customers](https://www.paloaltonetworks.com/customers?ts=markdown) * [Investor Relations](https://investors.paloaltonetworks.com/) * [Location](https://www.paloaltonetworks.com/about-us/locations?ts=markdown) * [Newsroom](https://www.paloaltonetworks.com/company/newsroom?ts=markdown) ## Popular Links * [Blog](https://www.paloaltonetworks.com/blog/?ts=markdown) * [Communities](https://www.paloaltonetworks.com/communities?ts=markdown) * [Content Library](https://www.paloaltonetworks.com/resources?ts=markdown) * [Cyberpedia](https://www.paloaltonetworks.com/cyberpedia?ts=markdown) * [Event Center](https://events.paloaltonetworks.com/) * [Manage Email Preferences](https://start.paloaltonetworks.com/preference-center) * [Products A-Z](https://www.paloaltonetworks.com/products/products-a-z?ts=markdown) * [Product Certifications](https://www.paloaltonetworks.com/legal-notices/trust-center/compliance?ts=markdown) * [Report a Vulnerability](https://www.paloaltonetworks.com/security-disclosure?ts=markdown) * [Sitemap](https://www.paloaltonetworks.com/sitemap?ts=markdown) * [Tech Docs](https://docs.paloaltonetworks.com/) * [Unit 42](https://unit42.paloaltonetworks.com/) * [Do Not Sell or Share My Personal Information](https://panwedd.exterro.net/portal/dsar.htm?target=panwedd) ![PAN logo](https://www.paloaltonetworks.com/etc/clientlibs/clean/imgs/pan-logo-dark.svg) * [Privacy](https://www.paloaltonetworks.com/legal-notices/privacy?ts=markdown) * [Trust Center](https://www.paloaltonetworks.com/legal-notices/trust-center?ts=markdown) * [Terms of Use](https://www.paloaltonetworks.com/legal-notices/terms-of-use?ts=markdown) * [Documents](https://www.paloaltonetworks.com/legal?ts=markdown) Copyright © 2026 Palo Alto Networks. All Rights Reserved * [![Youtube](https://www.paloaltonetworks.com/etc/clientlibs/clean/imgs/social/youtube-black.svg)](https://www.youtube.com/user/paloaltonetworks) * [![Podcast](https://www.paloaltonetworks.com/content/dam/pan/en_US/images/icons/podcast.svg)](https://www.paloaltonetworks.com/podcasts/threat-vector?ts=markdown) * [![Facebook](https://www.paloaltonetworks.com/etc/clientlibs/clean/imgs/social/facebook-black.svg)](https://www.facebook.com/PaloAltoNetworks/) * [![LinkedIn](https://www.paloaltonetworks.com/etc/clientlibs/clean/imgs/social/linkedin-black.svg)](https://www.linkedin.com/company/palo-alto-networks) * [![Twitter](https://www.paloaltonetworks.com/etc/clientlibs/clean/imgs/social/twitter-x-black.svg)](https://twitter.com/PaloAltoNtwks) * EN Select your language