Uncertainty Analysis

When calibrating a model we try to determine the simplest, most realistic, parameter field that is compatible with the data, without throwing information away by employing a parameter field that is too simple. Regularised inversion gives us the ability to do this.

But a calibrated parameter field is not reality - for reality is far more complex than this. To the extent that any prediction depends on the complexities of reality that cannot be represented uniquely in the calibrated model, that prediction has a potential for error - sometimes the potential for a lot of error. This potential for error must be accounted for when making management decisions on the basis of model outcomes.

Predictions that are especially prone to error are those that are highly sensitive to system detail. These include:

1. movement of pollutants in surface and ground water systems;
2. the interaction between ground and surface waters;
3. the response of an environmental system to extreme climatic events.

PEST's ability to accommodate parameterization complexity gives it the ability to estimate parameter sets of minimum error variance. It also allows it to characterize that error variance (or uncertainty). Traditional methods of parameter estimation based on pre-calibration parsimonization simply cannot do this.

PEST and its utilities allow a user to undertake comprehensive linear and nonlinear parameter and predictive uncertainty analysis as an adjunct to calibration based on highly parameterized inversion.

Linear analysis is approximate, but powerful. It allows characterization not only of parameter and predictive uncertainty. It can also calculate other useful quantities such as the following.

1. The contributions made by different individual parameters, or groups of parameters, to the uncertainty of predictions of interest.
2. The contributions to parameter and predictive uncertainty made by solution space and null space parameter components. The latter characterize uncertainty accrued through lack of information within a calibration dataset pertinent to a required prediction. The former characterize the contribution to predictive uncertainty made by measurement noise.
3. The worth of existing or new data in reducing predictive uncertainty. This can form the basis for optimization of data acquisition. Note that methods based on traditional parameter estimation which purport to achieve this same goal can be grossly misleading due to their inability to accommodate the (mostly dominant) null space contribution to parameter and predictive uncertainty.

Pre- (back row) and post- (front row) calibration contributions to the uncertainty of a key model prediction made by different groups of parameters.

PEST provides three options for nonlinear uncertainty analysis:

1. Predictive maximization/minimization subject to constraints on parameter reasonableness, and departures of specified model outputs from their field-measured counterparts.
2. "Predictive calibration" in which a prediction is "observed" to occur through a regularized inversion process that minimizes departures from a calibrated parameter field, and assesses the confidence level of the prediction through the size of this departure.
3. Calibration-constrained Monte-Carlo analysis.

The last of these is achieved through PEST's unique, and extremely powerful, null space Monte Carlo technique. This methodology allows a user to generate many different set of parameters, all of which are reasonable, and all of which calibrate a model. Each of these parameter fields may include any level of detail - well beyond that which can be represented uniquely in a calibrated model. If a prediction is made using many such calibration-constrained parameter fields, the uncertainty of that prediction can be assessed.

Null space Monte Carlo is extremely efficient. Once a model has been calibrated once, the numerical burden of re-calibrating it many times using parameter fields of arbitrary complexity is minimal.

A comprehensive tutoral on model predictive uncertainty analysis using PEST is available from the downloads page.