Scenario Testing and Sensitivity Analysis

Scenario testing is a way to compare what happens when assumptions change. A projection may show one result under a base case, another under a cautious case and another under a different retirement age, benefit start date or spending level. The result is conditional: it shows what happens under the assumptions shown, not what will happen.

Sensitivity analysis is more focused. It changes one input at a time, or one small group of inputs, to show which assumptions drive the result. This helps explain whether a projection is mainly sensitive to spending, inflation, return assumptions, fees, longevity, public-benefit timing, tax treatment or a rule threshold.

A deterministic range is not a probability range. If a calculator shows several scenario results, those outputs are what-if comparisons unless the model explicitly uses probability assumptions. A base case is a reference point, not automatically the most likely future.

Small changes can sometimes produce larger effects. This can happen when a change crosses a threshold, affects a tax or benefit calculation, shifts income into another year, compounds over many years, or changes the order of returns while withdrawals are occurring.

The useful question is not which scenario is right. It is what changed, whether the change is material, whether the assumptions are consistent, and what remains outside the model. The article How OpenBook Projections Work explains projection mechanics; Why Retirement Projections Change - and When to Update Them explains why projections change; this article explains how to compare those changes without turning them into predictions.

Table of contents

1. Introduction
2. From Understanding Projections to Comparing Them
3. What Scenario Testing Means
4. What Sensitivity Analysis Means
5. Scenario, Sensitivity, Stress Test, Range and Probability
6. How to Change One Assumption at a Time
7. When Assumptions Need to Move Together
8. Why Ranges Are Not Probabilities
9. Materiality: When a Changed Number Actually Matters
10. Why Small Changes Can Produce Large Effects
11. Sequence Paths During Retirement
12. Break-Even Points Are Not Recommendations
13. How to Read a Scenario Comparison
14. Common Misunderstandings
15. Final Thoughts
16. Key Takeaways
17. Important Notes

Introduction

A projection can make a future scenario look precise. It may show a projected balance, a shortfall year, a tax estimate or an income path down to the dollar. Precision is not the same as certainty.

A single projection can answer one question well. Scenario testing asks what happens when the question changes.

The value of a projection is not that it knows the future. Its value is that it makes relationships visible. It helps readers see how spending, income sources, inflation, investment returns, fees, taxes, benefits, longevity and timing interact under stated assumptions.

Scenario testing and sensitivity analysis build on that idea. They ask structured what-if questions. What changes if retirement begins earlier? What changes if inflation is higher? What changes if the same return set arrives in a different order? What changes if income crosses a benefit threshold?

The purpose is not to identify one personally correct answer. The purpose is to understand which assumptions matter, where the result is fragile, and whether two scenarios are being compared on the same basis.

From Understanding Projections to Comparing Them

How OpenBook Projections Work explains what a projection is: a conditional model built from facts, assumptions, source data, rules and limitations.

Why Retirement Projections Change - and When to Update Them explains why projections need to be refreshed when facts, assumptions, rules, household circumstances or the planning question change.

This article sits next to those articles. It explains how to compare projections deliberately. A changed projection may come from updated information. A scenario test asks what happens when one or more assumptions are deliberately varied. A sensitivity test asks which input is moving the result.

This distinction matters because not every changed output has the same meaning. Some changes reveal normal updates. Some reveal a threshold. Some reveal a fragile assumption. Some simply show that two scenarios are not comparable because several hidden inputs changed at once.

What Scenario Testing Means

Scenario testing compares complete sets of assumptions. A base case might use the current retirement date, current spending target and current public-benefit start ages. A second case might test earlier retirement with fewer saving years, earlier withdrawals and a different income bridge. A third case might test higher spending, lower returns or a delayed public-pension start date.

A scenario should be internally coherent. Changing only one number can be useful for diagnosis, but a complete scenario may need several linked inputs. For example, an earlier retirement scenario may affect employment income, savings, benefit timing, withdrawals and the number of years that assets must support spending.

A scenario is not a recommendation. It is a labelled comparison. Its usefulness depends on whether the changed assumptions are visible enough for the reader to understand why the result changed.

What Sensitivity Analysis Means

Sensitivity analysis is narrower. It usually changes one input at a time, then resets the model before changing another input. This helps isolate the effect of each assumption.

For example, a reader might test annual spending that is $5,000 higher, then reset the model and test a net return that is one percentage point lower, then reset again and test a planning horizon that is five years longer. The goal is to see which input moves the result most.

Sensitivity analysis is especially useful when a projection feels opaque. If a small spending change moves the result more than a return change, that is information. If a one-year retirement-date change moves the result more than expected, that is also information. The result points to a driver; it does not by itself say what to do.

Scenario, Sensitivity, Stress Test, Range and Probability

These terms are related, but they answer different questions. The table below gives the reader a compact interpretation map.

How to Change One Assumption at a Time

One-variable testing makes attribution clearer because only one assumption changes at a time. If the base case and the test case differ only in spending, the difference in output can be traced mainly to the spending input. If several assumptions change at once, the comparison may still be useful, but it no longer shows which input caused the change.

A practical sequence is:

• Start with a labelled base case and record the as-of date.
• Change one input, such as annual spending, inflation, return, fees, retirement age, public-benefit start age or planning horizon.
• Record both the changed input and the changed output.
• Reset to the base case before testing the next variable.
• Rank the variables by how much they changed the output that matters for the question being tested.

This method can reveal that the projection is more sensitive to one input than another. It can also show when a factor that seems important in conversation has little effect under the model's assumptions.

When Assumptions Need to Move Together

One-variable tests are excellent teaching tools, but real assumptions are often connected. A coherent alternate scenario may need several linked changes.

For example, inflation, interest rates, wage growth, fixed-income returns and borrowing costs should not be changed casually in isolation when the purpose is to build a coherent long-term scenario. FP Canada and the Institute of Financial Planning publish Projection Assumption Guidelines to support evidence-based long-term assumptions and emphasize documented, objective assumptions for projections.

Use sensitivity analysis to identify drivers. Use scenario analysis to understand how related assumptions interact.

Why Ranges Are Not Probabilities

A range can be helpful. A projection might show a base case ending balance of one amount, a lower-return case with a smaller balance and a higher-spending case with an earlier shortfall. That range shows how selected assumptions affect selected outputs.

But a deterministic range does not say how likely each outcome is. Unless the model uses stochastic methods and discloses probability assumptions, a range is a set of what-if comparisons, not a confidence interval.

This is an important OpenBook boundary. A projection can compare assumptions without pretending to know the probability of each future.

Materiality: When a Changed Number Actually Matters

Not every changed number deserves the same attention. A change is material when it could alter interpretation, update priority, risk discussion or a decision being compared.

A small numerical change may matter if it crosses a threshold, creates a cash-flow gap, changes benefit exposure, moves a shortfall into an earlier year or shifts the ranking of two scenarios. A larger-looking change may matter less if it only reflects rounding, a display convention or a future-dollar presentation.

Materiality is not one universal dollar amount. It depends on the question being tested. For one reader, a small OAS recovery-tax change may be important because it signals taxable-income sensitivity. For another, the larger issue may be whether essential spending remains covered in an early-market-decline scenario.

Why Small Changes Can Produce Large Effects

Many calculations appear smooth until they interact with thresholds, timing rules, required withdrawals, compounding or path dependence. This is why the misconception that a small input change always produces a small output change can be misleading.

Sequence Paths During Retirement

A sequence path is the order in which returns or events occur. If there are no contributions or withdrawals, the order of the same annual returns does not change the final compound value. With withdrawals, the order can matter because assets removed after weak years are no longer available to participate in later recovery.

This is especially relevant around the retirement transition. The years just before and just after retirement can be fragile because the household may be moving from contributions to withdrawals.

This example is simplified. It ignores tax, fees, inflation, deposits, account rules and benefit interactions. It is designed only to show how return order can matter when withdrawals occur.

Break-Even Points Are Not Recommendations

A break-even point can be a useful teaching tool. It shows where two scenarios become equal on one selected measure. For example, a pension timing comparison may have an age at which cumulative payments in one scenario catch up with another. A threshold test may show the income level at which a benefit recovery begins.

But a break-even point is not a complete answer. It may omit liquidity, taxes, survivor terms, GIS or OAS interactions, investment risk, spending needs, health uncertainty and household priorities.

OpenBook should use break-even points as comparison markers, not as advice labels.

How to Read a Scenario Comparison

A scenario table is most useful when it shows what changed, not only the final output. Two scenario outputs can differ because of one input, several inputs, data updates or a changed question. The comparison should make that visible.

When reviewing a scenario comparison, ask four questions:

• What changed? Identify the input, assumption, rule, date or household fact that differs from the base case.
• Is the comparison apples-to-apples? Confirm that dollar convention, tax year, inflation basis and source data are consistent.
• What is not modelled? Check whether the scenario excludes fees, tax details, market path, healthcare costs, legal issues, pension-jurisdiction rules or household changes.
• Is the change material? Decide whether the result changes interpretation, risk exposure, benefit exposure or the decision being tested.

The goal is not to react to every output. The goal is to understand what the output is saying and what it is not saying.

Common Misunderstandings

• A range of scenarios is a probability range. Unless probability assumptions are explicitly modelled, deterministic scenarios are what-if comparisons, not confidence intervals.
• The base case is the most likely future. A base case is a reference case. It may be reasonable, but it is not automatically the most likely.
• A small change in an input always produces a small change in the outcome. Thresholds, timing rules, compounding and sequence paths can amplify small changes.
• The average-return scenario captures investment risk. When withdrawals occur, the order of returns matters. A smooth average path can hide sequence risk.
• More scenarios make the model more accurate. More scenarios can improve understanding only if they answer useful questions and use coherent assumptions.
• A stress test is the true worst case. A stress test is only the adverse case selected. It does not prove that worse outcomes cannot occur.
• Monte Carlo probabilities are objective truth. Stochastic outputs depend on the model's probability assumptions, distributions, correlations and tax/benefit logic.
• Sensitivity testing is only about investment returns. Spending, inflation, fees, retirement age, longevity, tax thresholds and benefit timing can be equally or more important.
• Changing several assumptions at once proves which one caused the result. If multiple assumptions change, attribution is unclear unless the comparison is structured.

Final Thoughts

Scenario testing and sensitivity analysis are not ways to make a projection certain. They are ways to make uncertainty easier to examine.

A useful scenario does not say what the future will be. It says what changes when a defined set of assumptions changes. A useful sensitivity test does not choose a strategy. It shows which input is moving the result.

That is why this article belongs with the two earlier projection articles. First, understand that a projection is conditional. Then understand why projections change. Then use scenarios and sensitivity tests to compare assumptions deliberately.

A projection becomes more valuable when it stops being a single number and becomes a transparent comparison that helps readers understand why different outcomes are possible.

Key Takeaways

• Scenario testing compares complete sets of assumptions.
• Sensitivity analysis changes one input at a time to identify which assumptions drive the result.
• A deterministic range is not a probability range unless probability assumptions are explicitly modelled.
• A base case is a reference case, not automatically the most likely case.
• Small input changes can produce larger output changes near thresholds, timing rules, required withdrawals, compounding effects or sequence paths.
• One-variable tests help with attribution, while coherent scenarios may need several linked assumptions.
• Sequence risk appears when investment returns and cash flows occur together.
• Break-even points are useful comparison markers, but they are not recommendations.
• The most useful scenario comparison shows what changed, whether it is material, and what remains outside the model.

Important Notes

This article is educational only. It does not provide financial, tax, legal, accounting, investment, retirement, pension or other professional advice.

Scenario and sensitivity examples are simplified to explain mechanics. They do not determine an individual's retirement readiness, tax result, benefit eligibility, withdrawal strategy, pension timing or suitable investment approach.

Tax, benefit, pension, registered-account and public-program rules can change. Current figures, thresholds, program details and source links should be checked against official sources before publication and before calculator data is updated.

A deterministic projection does not reproduce all possible market paths, inflation shocks, health or care costs, legal issues, pension-plan details, household changes or future law changes unless those items are specifically modelled.