We modeled the Cybersecurity Canon after the Baseball or Rock & Roll Hall-of-Fame, except for cybersecurity books. We have more than 25 books on the initial candidate list, but we are soliciting help from the cybersecurity community to increase the number to be much more than that. Please write a review and nominate your favorite.
The Cybersecurity Canon is a real thing for our community. We have designed it so that you can directly participate in the process. Please do so!
Book Review by Canon Committee Member, Steve Winterfeld: How to Measure Anything in Cybersecurity Risk (2016) by Douglas W. Hubbard and Richard Seiersen
How to Measure Anything in Cybersecurity Risk is a book that reads like a college statistics textbook (but the good kind you highlight a lot). It is a book anyone who is responsible for measuring risk, developing metrics, or determining return on investment should read. It is grounded in classic quantitative analysis methodologies and provides a good balance of background and practical examples. This book belongs in the Cybersecurity Canon under Governance Risk and Compliance (GRC).
As I said, this book reads like an education in quantitative modeling and how to apply the methodology to cybersecurity. It truly challenges the current common practices in use to develop expert opinion-based risk frameworks. Here is a snippet from the book:
"So let’s be clear about our position on current methods: They are a failure. They do not work. A thorough investigation of the research on these methods and decision-making methods in general indicates the following: There is no evidence that the types of scoring and risk matrix methods widely used in cybersecurity improve judgment. On the contrary, there is evidence these methods add noise and error to the judgment process. Any appearance of “working” is probably a type of “analysis placebo.” That is, a method may make you feel better even though the activity provides no measurable improvement in estimating risks (or even adds error). There is overwhelming evidence in published research that quantitative, probabilistic methods are effective. Fortunately, most cybersecurity experts seem willing and able to adopt better quantitative solutions. But common misconceptions held by some—including misconceptions about basic statistics—create some obstacles for adopting better methods. How cybersecurity assesses risk, and how it determines how much it reduces risk, are the basis for determining where cybersecurity needs to prioritize the use of resources. And if this method is broken—or even just leaves room for significant improvement—then that is the highest-priority problem for cybersecurity to tackle!”
The authors lay out the book in three sections:
The cybersecurity community suffers from not having standard evaluation metrics, like earnings before interest, taxes, depreciation and amortization (EBITDA). The authors try to bring some discipline to terms by offering standard definitions coming from the quantitative analytics field. From the book:
They also walk the reader through established methodologies like: Monte Carlo simulations, Bayesian interpretation, risk matrix, loss exceedance curve, heat maps, chain rule tree, beta distribution changes, regression model predations, analytics maturity mode, power law distribution, subjective probability, calibration, dimensional modeling, expected opportunity loss, bunch of guys sitting around talking, expected value of prefect information, NIST and ISO. They explain how, in Excel, so they are truly practical. They also lay out survey results from attitudes toward quantitative methods, global information security workforce study, and stats literacy and acceptance studies.
This work follows other work like Factor Analysis of Information Risk (FAIR) which is a well-recognized value at risk (VaR) framework. They outline another Monte Carlo–based methodology and tools like those developed by Jack Jones and Jack Freund. Another similar work is The Wisdom of Crowds by James Surowiecki.
Finally the book has some great online resources. You can find eight sample downloads of the methods explained, as well as webinar/blog info.
How to Measure Anything in Cybersecurity Risk is an extension of Hubbard’s successful first book, How to Measure Anything: Finding the Value of “Intangibles” in Business. It lays out why statistical models beat expertise every time. It is a book anyone who is responsible for measuring risk, developing metrics, or determining return on investment should read. It provides a strong foundation in qualitative analytics with practical application guidance.
Bottom line: The authors lay out a solid case for why other industries with the similar challenges of lack of quantifiable, standardized or historical actuarial table-like data are able to use classic statistical modeling and methodologies to measure risk in a qualified, repeatable way. Definitely worth considering.