Uncertainty Quantification
Goals
Go to IntroductionThis experiment will use the model of Pine Island Glacier that was saved in the previous Pine Island Glacier modeling tutotial. It aims to use the ISSM-Dakota integrated model system to (1) quantify the uncertainties of model output in response to errors in model input and (2) quantify sensitivities of model output to spatial perturbations in model input.
Our Uncertainty Quantification (UQ) methods are based on the Design Analysis Kit for Optimization and Terascale Applications (Dakota) software [Eldred2008], which is embedded in ISSM. The following diagram illustrates the relationship between ISSM and Dakota. The ISSM mesh must be partitioned (i.e. vertices can be grouped together so that Dakota varies them together - this is helpful when you want to vary equal areas over the unstructured mesh). To partition the mesh, you can do so linearly (one partition per vertex), or you can use an external package software like Chaco to weight vertices and create the partitions you desire. Dakota is resposible for varying the provided inputs in the user-defined way (uniform, normal, etc.) for each mesh partition and then launching an ISSM run with the perturbed forcing. Dakota is also responsible for creating statistics for output, which are also user defined. Output diagnostics include ice mass flux through defined gates and scalar output (e.g. Ice Volume, Total SMB, etc.). Tutorial steps to be taken:
Samping Analysis: Quantify the uncertainties of model output (diagnostics like mass flux, Ice Volume, Max Velocity) in response to errors in model input. The figure below illustrates an example of Sampling errors in ice thickness. The result for each gate, is a histogram of Mass Flux (one value per each model run, or sample). Below is the resulting histogram for mass flux gate 2. Sensitivity Analysis: Quantify sensitivities of model output to small spatial perturbations in model input. The figure below illustrates how this is accomplished. One by one, partition input is changed by a small percentage, and a model run is launched. For this specific run, changes in model diagnostics (output) are assessed by Dakota. This is done for each partition, such that the number of model runs is equal to the number of mesh partitions. In the end, every diagnostic is associated with a sensitivity value at every partition. In this way, we can make a map of sensitivities for each diagnostic. Sensitivities can also be ranked, for each diangistic, in importance. One such example of Dakota output is the 'importance factor', or sensitivities scaled by error margins [Larour2012a, Larour2012b], illustrated below as UQ sensitivity analysis output for mass flux gate 2. For manuscript examples of these studies, see [Larour2012a,Larour2012b,Schlegel2013,Schlegel2015]. Flux GatesFlux gates are ARGUS ( Mass fluxes will be computed in (Gt/yr) for all of these gates (using the depth-average ice velocity, ice thickness, and ice density). Run step 1 of the Loading Cross-Over ErrorsFor ice thickness errors we will use McCords cross-over errors from CReSIS. First you will load errors. Some of these errors are too large, too small, or need to be interpolated onto a larger domain (you will filter these out). Load cross overs Run Step 2 in the Sampling AnalysisIn order to accomplish the sampling step, we must first partition the mesh into equal area partitions. We'll start with 50. You can try and play with the package for partitioning ('chaco' or 'linear'), the number of partitions, and weighting ('on' or 'off'):
To plot the corresponding partition over a plot of the mesh:
Note that after using Chaco, your partitions may look different from those illustrated here, because there is a randomness to the Chaco algorithm, and results differ on different computer systems. Second, we must define our UQ input. Here, we will sample ice thickness (H), so we must define errors on each partition for H with a corresponding PDF (Probability Density Function). Here we calculate the crossover errors on each partition. In this example, we will sample a normal error distribution around every partition. To do so, we need to specify to Dakota that we want a normal sampling, and we must provide the standard deviation of error at every partition. Because crossover errors represent the full range of thickness errors, we assume this represents a 6-sigma normally distributed spread. Therefore, we set the standard deviation equal to the crossover error at a particular location, divided by 6:
Third, we must set up the desired diagnostics, or output responses. In this case, we choose ice mass flux at 13 flux gates around the domain:
For all responses, we specify a string identifier and the desired output confidence intervals. We also need to specify an
Finally, we need to designate a sampling strategy. Options include
In addition, we setup persistent parameters, this includes parallel concurrency, verbosity, and data backup:
We also have to tighten the solver tolerance (in order to avoid spurious sensitivities to develop) before solving:
Because the ISSM-Dakota framework now runs in parallel, our implementation requires that Dakota runs with a master/slave configuration. This means that at least 2 CPUs are needed to run the UQ, such that: md.cluster.np=md.qmu.params.processors_per_evaluation*N
where
Don't forget to deactivate inversion (
Note that results will be in Sensitivity AnalysisNext we quantify importance factors (sensitivities scaled by error margins) for model inputs: ice thickness (H), basal friction (), and ice rigidity (B). We specify a 5% error margin on all inputs. For partitions, we choose 10 partitions, and setup for model diagnostics is the same as for sampling analysis.
We specify the same parallel CPU configuration, and we solve the same way as in step 3. Note this time, we turn Dakota verbosity on as an example:
Run step 5 to launch the sensitivity runs. Plot ResultsPlot Sampling Results: In order to plot the results, we extract the results for one of the mass flux gates, and display a histogram of the sampling results for that particular gate. ISSM has a plotting function for this,
Plot Sensitivity Results:
Additional Exercises
References
|