Model selection theory: the need for a more nuanced view on use-novelty

Abstract

Climate policy needs to be informed by the results of the best climate models, with respect to the issue at hand. To evaluate climate models, it is essential that the best available methods for confirmation are used. A hotly debated issue on confirmation in climate science (as well as in philosophy) is the requirement of use-novelty (i.e. that data can only confirm models if they have not already been used before, e.g. for calibrating parameters).

This paper investigates the issue of use-novelty in the context of the mathematical methods provided by model selection theory. We will first argue that model selection theory provides a good method for evaluating many climate models. Then we will show that the picture model selection theory presents us with about use-novelty is more subtle and nuanced than the commonly endorsed positions by climate scientists and philosophers. More specifically, we will argue that there are two main cases in model selection theory.

On the one hand, there are the methods such as cross-validation where the data are required to be use-novel. On the other hand, there are the methods such as Bayesian confirmation or the Akaike Information Criterion (AIC) for which the data cannot be use-novel. Still, for some of these methods (like AIC) certain intuitions behind the use-novelty approach are preserved: there is a penalty term in the expression for the degree of confirmation by the data because the data have already been used for calibration.

The common positions argued for in climate science and philosophy are either that data should always be use-novel or that the use-novelty criterion is irrelevant. According to model selection theory these positions are too simple: whether or not data should be use-novel depends on the specific method used. For certain methods data should be use-novel, but for others they cannot and thus need not be use-novel.

Where & when?

Wed 14 January 2015, 3.15 - 5pm
Faculty of Philosophy, Room Omega