Rating the Security Ratings Vendors

Editor's note: Expanse has since been rebranded as Cortex Xpanse. 

Today, security rating systems have become a growing part of the security landscape. Used by insurance firms as well as vendors to gauge cyber risk, security rating fills an understandable void by answering, “How secure are organizations I can’t see?” Last year, then-Expanse CEO Tim Junio (and now Palo Alto Cortex SVP of products) weighed in on security rating and pointed out the same basic flaw we saw with security risk ratings: “the data is terrible.” Tim’s year-old observation still holds true for several reasons.

Almost exactly a year later in February 2021, Forrester published “The Forrester New Wave™: Cybersecurity Risk Ratings Platforms, Q1 2021.”  According to the report, "Cybersecurity Risk Ratings Are Not Yet Ready For Prime Time."

First, let’s take a step back and understand what exactly is a security rating? How are they produced? Who do they benefit? A security rating is a score and in some cases a grade that represents and quantifies an organization's overall cybersecurity risk level, indicating how vulnerable the organization is to threat actors. Security rating organizations use various external data (e.g., registration data, basic internet scanning) to evaluate an organization and unfortunately in most cases only touch the surface of an organization’s cybersecurity landscape.

However, these security scorecards allow comparison between organizations and their vendors, acting as effective eye candy for executives and board of directors since they provide an easily digestible snapshot of the organization’s cybersecurity. Although security scorecards seem to provide some insights, they fall short of identifying an organization’s complete attack surface to give a misleading cyber security summary

There are three major reasons that security scorecards fall short in delivering an accurate picture of an organization’s cybersecurity landscape.

Reason 1: Security scoring has a high rate of false positives and misattribution

Many security scorecard organizations use a “mile-wide and inch-deep” approach to gather external information that their scores are based on. Unfortunately, this approach results in an extremely high false positive rate and inaccurately depicts the risk exposure of an organization. One reason: weak attribution algorithms that pollute an organization’s asset map with false information to superficially inflate risk. This attribution algorithm is similar to a baked credit score. Instead of basing your score on just financial information related to you, the credit score instead includes information from everyone with your name or a pretty close match. This results in receiving a credit score based on everyone around the world with your same name. If your name is Taylor Swift, you’re pretty much guaranteed a strong credit rating.

Reason 2: Security scoring is based on data that is stale, outdated, and unreliable

In order to model your risk exposure across your network, active services must be running in order for an attacker to exploit. Many security scorecards claim they scan the entire IPv4 space on a weekly cadence or depend on third party data that’s updated monthly to identify active services running on your network. However, adversaries are agile and resilient. They can index the entire internet in under an hour for a given protocol, allowing them to quickly find and exploit these vulnerabilities. With cloud assets being ephemeral and your attack surface changing instantaneously, monthly data refreshes are not sufficient to give you an accurate picture of the assets across your organization and oftentimes up to 50% of your network can be completely missed. By the time you receive your security rating, they are already significantly out of date. During the recent OWA vulnerability episode, Palo Alto Networks used Expanse to identify the global footprint of unpatched assets within hours of discovery.

Reason 3: A “good” score doesn’t indicate the organization is secure

Many organizations use security ratings to obtain an arbitrary number to present to executives and board of directors to showcase how their security program is performing. However, these scores are not based on industry frameworks (e.g., NIST CSF, CISSP) but rather a subjective number which lacks the ability to drive strategic initiatives and power to predict a cybersecurity breach. If you receive a “B” what does this mean for your cybersecurity program? What’s the likelihood of a cybersecurity breach at your organization? What areas of my organization are least secure? Security scorecards are not able to answer these types of questions and unfortunately many organizations depend too highly on these arbitrary scores and are blindsided when an attack happens. Organizations simply can’t improve their security posture without data about specific issues.

From the reasons above, security ratings are not an ample way to determine the risk landscape of an organization. They misconstrued the effectiveness of your cybersecurity program and leave you with an incomplete picture of your cybersecurity risks. So what do you get from a security scorecard? A subjective number and inaccurate representation of your risk landscape that you can present to your executive leadership - that’s it! How can organizations obtain actionable insights to drive their cybersecurity maturity and reduce their risk landscape?

Understanding your Attack Surface and Using Data Driven Results to Protect Your Organization

Attack surface is everything that is an adversary's playground–the assets that attackers can easily exploit on the public internet. Attack surface is an organization’s digital presence across the internet - this includes leased IP space from AWS, an employee’s laptop that has proprietary information on it, and even your third party suppliers’ assets. These are all areas where an attacker could compromise your network and cause you millions of dollars in remediation effort. An organization needs to inventory all of these assets and ensure controls are in place to protect against adversaries.

Attack surface represents your adversary's playground – the assets that attackers can easily find and exploit on the public internet which aside from corporate owned address space, includes leased IP. space from AWS, an employee's laptop that has proprietary information on it, and even your third party suppliers' assets.

For more information on identifying and managing strategic supplier risk, download our white paper.