I Was Wrong about Risk Assessments—and You Probably Are as Well

Last Updated on January 14, 2024

Einstein once said, “The more I learn, the more I realize how much I don’t know.” Unfortunately, I have come to that realization not just once, but about 758 times.  As I was reviewing/tuning our company’s Risk Assessment in preparation for our ISO 27001 surveillance audit, I realized I had hit #759.
In explaining Risk Assessment/Analysis, I often try to get clients to think of it as a “risk register.” In a previous blog post, I extolled the value of this concept in helping people grasp the purpose and value of risk assessment.
The problem with the “risk register” view is that it carries an expectation that as “new” risks arise they are added to the register, and as risks are addressed they are “closed” and leave the register.  I know the vast majority of our clients look at it that way. I largely did as well.

There Are No “New Risks,” Only New Contexts

We recently changed our Risk Assessment methodology in a way that ensures that every “threat agent” is analyzed against every “vulnerability” in clients’ respective libraries (lists).  As I was reviewing our Risk Assessment I suddenly had the realization there are no “new risks.” There are only new contexts that alter existing risks. This, IMHO, has a significant impact on ongoing risk assessment.  I know that sounds a little obtuse, so let me try to explain.
By definition, a risk measures the impact of an event based on the probability that a threat agent can impact a vulnerability. If you have a “complete” threat agent and vulnerability library, you are assessing every potential risk that you face. So, there are no truly “new” risks; there is only the perception of new risk.
How do these “new” risks arise?  Simple—context changes. For example:

• Changes in laws, regulations, and contractual obligations
• Changes in the vendors you use, and the services existing vendors provide to you
• Changes in the services you provide to your clients
• Changes in management’s expectations
• Changes in your cyber liability insurance
• Changes to buildings or networks
• Changes from moving an application from on-premises to the cloud

This idea of context is inherent in all major information security risk management frameworks. In ISO 27001, for instance, the notion of context is referred to as scope determination (or scoping). NIST 800-30 refers to it as system characterization. Octave refers to it as Asset Profiling.
Assuming I am right (and for the record, I am :>)), what then?

How This Affects Risk Assessments

The takeaway is simple. Risks don’t ever leave your risk assessment. While they may not need treatment now based on today’s context, they may need treatment next month, based on that context.
This alters Risk Assessment/Analysis/Treatment in two fundamental ways:

• Rather than eliminate risks from your risk register that don’t currently require any treatment, you should instead just denote the next time that they need to be reviewed against context.
• Rather than conducting a Risk Assessment annually, you should instead conduct a scoping exercise, determine how changes in your context/scope have changed existing risks, and then update your Risk Assessment/Register/Plans accordingly.

A final point: Most security practitioners tend to think that the maturity/effectiveness of a control largely stays the same or improves year over year. When you take a context change centric approach, it becomes more apparent how context changes can notably increase or reduce the maturity/effectiveness of controls.

Case Studies

A couple of examples from our risk assessment to illustrate my thoughts:

• We have mature patch and configuration management and vulnerability management controls that we had scored at a 3/5 (Defined) and 4/5 (Managed), respectively. The controls do a great job managing the risk of a Hostile Outsider trying to attack weaknesses in externally facing systems. This past year we deployed a test system on Amazon Web Services (all our systems of note other than that are deployed in an Iron Mountain colocation facility) that was not being properly scanned because our processes were tied specifically to the IP addresses that are directly allocated to Pivot Point Security.  Fortunately, that system does not contain any production data and its configuration kept it secure. Reviewing the same risks, with different contexts, made it simple to identify this issue (a “new old” risk, if you will).  Further, we realized that the context change had changed the maturity and effectiveness of our vulnerability management control and required changes in our policy and process.
• Conversely, we had a solid control that addressed the risk of any threat agent exploiting weaknesses in our “system reuse and/or disposal practices.” We had designed these controls before we started encrypting the hard drives of all systems, so they were a bit onerous. They required drives to be removed from systems, stored in a secured area, and shredded twice per year by a third-party asset recovery firm. When we revisited context, we realized that because all our drives were now encrypted, we could reduce the extent and rigor of some of our controls, as this “old new” risk was much lower than it had been. We chose to reduce the rigor of the control, which saved us a fair amount of time and money.

I hope you found the idea that there are no “new risks”, only context changes that re-open “closed” risks or increase/decrease “open” risks, as interesting and valuable as I did.
It’s now 10PM, and time to call it a day. I guess “realization #760” will have to wait until at least tomorrow… fortunately the Knob Creek won’t …