Natural Variability, Attribution and Climate Models #11
Estimating natural variability using climate models
In #9 we looked at an interesting paper (van Oldenborgh and co-authors from 2013) assessing climate models. They concluded that climate models were over-confident in projecting the future, at least from one perspective which wouldn’t be obvious to a newcomer to climate.
Their perspective was to assess spatial variability of climate models’ simulations and compare them to reality. If they got the spatial variation reasonably close then maybe we can rely on their assessment of how the climate might change over time.
Why is that?
One idea behind this thinking is to consider a coin toss:
If you flip 100 coins at the same time you expect around 50 heads and 50 tails. Spatial.
If you flip one coin 100 times you expect 50 heads and 50 tails. Time.
There’s no strong reason to make this parallel with climate models on spatial and time dimensions but climate is full of challenging problems where we have limited visibility. We could give up, but we just have the one planet so all ideas are welcome.
In the paper they touched on ideas that often come up in modeling studies:
assessing natural variability by doing lots of runs of the same climate model and seeing how they vary
comparing the results of different climate models
This wasn’t the focus of their paper but they did briefly discuss the results.
The results indicate:
..more than half the ensemble spread is due to the model estimate of natural variability and less than half due to model spread at this location
This idea often shows up in climate papers - we can estimate natural variability by looking at many runs from the same climate model and seeing how much they vary. And by looking at the variation between models.
This might be true if climate models were close to perfect representations of actual climate, and if we could run enough simulations with:
Changes in initial conditions to cover our uncertainty about actual values
Changes in all the parameters about which we have uncertainty
We’re a long way from this ideal.
Here’s an example to demonstrate the problem. This graphic shows four climate models each under two scenarios (RCP4.5 & RCP8.5) of CO2 emissions. We’re looking at the change across a century in rainfall in Australia (100% means no change):
The MIROC climate model (top right) says “lots more rainfall” while the CSIRO model (bottom right) says “lots less rainfall”.
Perhaps this is simply showing us the wide range of climate variability under more CO2 and all these outcomes are equally likely.
Or perhaps climate models aren’t very good at simulating rainfall.
References
Reliability of regional climate model trends, GJ van Oldenborgh et al, Environ. Res. Lett. (2013)
Here's AR6 (IPCC 6th assessment report) making a valuable comment re the rainfall example I raised in the article. From ch 3, p449:
"A fact hindering detection and attribution studies in precipitation and other hydrological variables is the large internal variability of these fields relative to the anthropogenic signal."
Here's a comment from "Taking climate model evaluation to the next level", Veronika Eyring et al, Nature Climate Change (2019):
An important issue that remains to be fully addressed is the extent to which model errors affect the quality of climate projections and subsequent impact assessments. Traditionally, many climate projections are shown as multimodel averages in the peer-reviewed literature and IPCC reports, with the spread across models presented as a measure of projection uncertainty.
There is now emerging evidence that weighting based on model performance may improve projections for specific applications.
A further complication in devising model weighting approaches is that many CMIP models share components, or are variants of another model in the ensemble, and hence are not truly independent.
This has the potential to bias the multimodel results in ways that are only beginning to be explored.
The lack of independence challenges the notion of a ‘model democracy’, in which each model is weighted equally.