Episode 52 — Choose Qualitative Versus Quantitative Risk: When Each Method Actually Helps
In this episode, we’re going to tackle a choice that shows up in almost every OT risk program, even if nobody names it out loud: do we describe risk with words, or do we calculate it with numbers. Qualitative risk methods use categories like high, medium, and low, while quantitative methods try to estimate risk with numerical values like probabilities, dollar impacts, or expected loss. If you’re new to cybersecurity, it can feel like quantitative must be better because it sounds more scientific, and qualitative must be weaker because it sounds like opinions. In OT, that assumption can lead to frustration, because numbers can create a false sense of precision when the underlying data is uncertain, and words can be dismissed as vague when they are not tied to clear definitions. The reality is that both methods can be helpful, and the best choice depends on what decision you need to make, what data you have, and how much effort you can invest without slowing real progress. OT adds extra constraints because systems are complex, consequences can be physical, and many variables are hard to measure directly. The goal here is to help you understand what each method does well, what it does poorly, and how to choose a method that leads to better decisions rather than better spreadsheets.
Before we continue, a quick note: this audio course is a companion to our course companion books. The first book is about the exam and provides detailed information on how to pass it best. The second book is a Kindle-only eBook that contains 1,000 flashcards that can be used on your mobile device or Kindle. Check them both out at Cyber Author dot me, in the Bare Metal Study Guides Series.
A qualitative approach is essentially structured judgment, and that structure is what makes it useful. Instead of guessing informally, you define categories for likelihood and consequence and then combine them into a risk rating. The strength of qualitative methods is that they can work even when you lack detailed numerical data, which is common in OT. You can still make good comparative decisions by ranking scenarios, identifying high-criticality assets, and deciding which mitigations to prioritize first. Qualitative methods also support communication, because most stakeholders can understand what high risk means if the category definitions are clear and consistent. Beginners sometimes think qualitative means subjective, but the method becomes much less subjective when categories are defined with real thresholds, like maximum tolerable downtime or potential for safety impact. Another advantage is speed, because qualitative assessments can produce actionable results without months of data gathering. In OT, speed matters because the environment changes and risks do not pause while you perfect your model. Qualitative methods can also adapt to safety and operational realities, because they allow you to incorporate expert knowledge about process behavior, maintenance constraints, and realistic failure modes. When done carefully, qualitative risk is not vague, it is practical.
Quantitative risk methods aim to produce numerical estimates that can be compared across options, often with the hope of supporting cost-benefit decisions. In theory, quantitative methods can help you answer questions like how much loss we expect per year from certain scenarios, or how much value a mitigation provides by reducing expected loss. This can be useful when leadership must allocate budgets across many competing needs, and they want to compare security investment to other kinds of investment. In OT, quantitative methods can also help when you have measurable consequences like downtime cost per hour, repair costs, or product loss costs. Beginners often find quantitative methods appealing because numbers feel authoritative, but the key issue is that the number is only as good as the assumptions behind it. If you cannot estimate likelihood with reasonable confidence, your expected loss calculations can be more misleading than helpful. Quantitative methods also take more time and expertise, because you need to model distributions, uncertainty, and dependencies rather than just picking a single number. In OT, those dependencies can be complex, and data may be limited, especially for rare but high-impact events. Quantitative methods can be powerful, but they are not automatically superior, and using them badly can create false confidence.
The most important practical question is what decision you are trying to support, because the decision determines what method actually helps. If you are trying to decide which ten risks to address first, or which assets deserve the strongest controls, qualitative methods often work well because you need ranking and prioritization more than precise valuation. If you are trying to justify a major investment, like a segmentation redesign or a recovery modernization project, quantitative analysis might help because it can translate risk into financial terms that leadership understands. If you are trying to compare two competing mitigations, like investing in monitoring versus investing in backup modernization, a quantitative view might help if you can reasonably estimate the difference in consequence reduction. Beginners should understand that risk methods are tools, not identities, and the goal is not to pick a method that sounds impressive, but to pick a method that reduces uncertainty for the decision at hand. In OT, many decisions involve feasibility and safety constraints, and qualitative methods can incorporate those constraints explicitly. Quantitative methods can still include constraints, but they require careful modeling and often still end up relying on expert judgment. A good choice starts by stating the decision and the audience.
Data availability is the next deciding factor, because quantitative methods require inputs that are often hard to obtain in OT. You need a way to estimate likelihood, which might require incident data, exposure data, control effectiveness data, and threat intelligence, and those inputs may be incomplete. You also need a way to estimate consequence, which might include downtime costs, safety impacts, environmental impacts, and reputational harm, and not all of those translate cleanly into dollars. Even when you can estimate downtime cost, the cost can vary depending on timing, season, and supply chain context. Beginners sometimes assume that if you cannot measure something precisely, you cannot measure it at all, but qualitative methods allow you to use reasoned categories with clear definitions. Quantitative methods can incorporate uncertainty through ranges and distributions, but you still need some data to anchor those ranges. In OT, some data exists, like maintenance records, downtime logs, and production losses, but cyber-specific likelihood data may be sparse. That does not mean quantitative is impossible, but it means it must be treated as estimation with uncertainty, not as exact truth. The more uncertain your inputs, the more careful you must be about how you present the results.
Another factor is the risk of false precision, which is one of the biggest dangers for beginners when they encounter quantitative risk. A model might output a number like 2.3 million dollars of expected annual loss, and that number can look authoritative even though it may be built on rough assumptions. In OT, where rare events can have outsized consequences, small changes in assumed likelihood can swing the result dramatically. If leaders treat the number as exact, they may make overconfident decisions, or they may argue about the number instead of focusing on the underlying risk drivers. Qualitative methods can also be misused, such as labeling everything high, which removes meaning, but qualitative categories are less likely to create an illusion of precision. The best practice is to match the precision of your output to the confidence of your inputs. If you only have rough estimates, you should not present results with more precision than the data supports. Beginners should learn to ask, how confident are we in this estimate, and what assumptions drive it. When you focus on assumptions and confidence, you keep the risk conversation honest.
A qualitative method becomes much more effective when category definitions are tied to OT realities rather than generic language. For likelihood, you might define categories based on exposure and plausibility, such as whether a pathway is always on, whether authentication is strong, and whether monitoring is present. For consequence, you might define categories based on safety impact, downtime thresholds, equipment damage potential, and environmental implications. The important part is that categories are defined in advance and used consistently, so different assessors do not label the same scenario differently based on mood or bias. Beginners sometimes assume qualitative is just a meeting where people vote, but a structured qualitative method can be quite disciplined. It can include checklists of factors that push likelihood up or down and consequence up or down, making the result more repeatable. It can also include a way to capture rationale, so you record why a scenario was rated high and what evidence supports that rating. This rationale is valuable because it guides mitigation choices and makes reassessment easier later. When qualitative risk is structured and documented, it becomes a reliable decision tool.
Quantitative methods can be introduced gradually, and that is often the most realistic approach in OT. Instead of trying to quantify everything, you can quantify parts of consequence where data exists, like downtime cost ranges, repair costs, and replacement lead times. You can then keep likelihood qualitative or semi-quantitative, using ranges rather than exact probabilities. Over time, as monitoring improves and as incident data accumulates, you can refine the likelihood estimates. Beginners should understand that quantitative does not have to mean exact, it can mean using numbers where they help and using ranges where uncertainty is high. This hybrid approach can support better decisions because it provides financial context without pretending that cyber likelihood is precisely known. It also helps communicate with leadership because numbers can translate impact into a language used for budgeting and planning. At the same time, the hybrid approach keeps the assessment grounded in operational reality by acknowledging uncertainty. In OT, that honesty is essential because overconfidence can lead to unsafe decisions. Quantitative methods should serve the decision, not dominate it.
A practical way to decide between methods is to consider how the results will be used and challenged. If the results will be used to prioritize operational improvements and coordinate across teams, qualitative ratings with clear rationale may be most effective because they are easier to explain and act on. If the results will be used to justify investment to finance or to compare across very different risks, some quantification may be helpful, but only if the organization accepts uncertainty and does not demand false precision. You should also consider whether the method will be sustainable, because a method that takes six months to run may not be repeatable, and an assessment that cannot be repeated will not track progress. Beginners sometimes think the best method is the most complex, but in practice the best method is the one that the organization can apply consistently and improve over time. OT environments benefit from repeatable assessments because systems change, and risk changes with them. A simpler method applied regularly can outperform a complex method applied rarely. Sustainability is part of usefulness.
Finally, choosing qualitative versus quantitative risk in OT is about matching the tool to the reality of the environment and the needs of decision-makers. Qualitative methods are often the backbone because they support prioritization under uncertainty and align well with operational and safety context. Quantitative methods can add value when you have reliable data for parts of consequence or when a decision requires financial framing, but they must be used with humility about uncertainty. The most mature approach is not choosing one forever, but building a risk program that can use qualitative structure to stay consistent while gradually adding quantitative elements where they genuinely improve decisions. For beginners, the key takeaway is that risk assessment is not a contest to produce the most impressive numbers, it is a discipline to reduce uncertainty and choose meaningful mitigations. When you keep that purpose front and center, you will naturally choose methods that help rather than methods that distract. In OT, where safety and reliability are always in view, that practical focus is what turns risk assessment into real protection.