Visualization

ClimaCalibrate provides plotting utilities for plotting the mean forward map evaluation, columns of the G ensemble matrix, and the true observation via a Makie extension.

To plot the mean forward map evaluation, columns of the G ensemble matrix, and the true observation, you can use Visualization.plot_g_mean, Visualization.plot_g, and Visualization.plot_obs respectively. The mutating versions also exist as Visualization.plot_g_mean!, Visualization.plot_g!, and Visualization.plot_obs!. All plotting functions takes an EKP.EnsembleKalmanProcess object to plot from. Additionally, the plotting function accept an iter keyword argument for plotting from a specific iteration. If the keyword argument is not provided, then the last iteration is used for plotting. You can expect all keyword arguments that work with Makie.Lines to also work with these plotting functions and that the plotting functions behave like Makie plotting functions.

Example

Here is a complete example where we use the plotting functions to plot the ensemble members, the mean forward map evaluation, and the true observations from the second iteration.

import ClimaCalibrate
# To use this extension, one of the Makie backends should be loaded
import CairoMakie

fig = CairoMakie.Figure()
ax = CairoMakie.Axis(
    fig[1, 1],
    title = "G ensemble members, mean forward map evaluation, and observations",
    xlabel = "Index",
    ylabel = "Value",
)
g_plot = ClimaCalibrate.Visualization.plot_g!(
    ax,
    ekp;
    iter = 2,
    color = :black,
    alpha = 0.2,
)
g_mean_plot =
    ClimaCalibrate.Visualization.plot_g_mean!(ax, ekp; iter = 2, color = :black)
obs_plot =
    ClimaCalibrate.Visualization.plot_obs!(ax, ekp; iter = 2, color = :blue)

CairoMakie.Legend(
    fig[1, 2],
    [g_plot, g_mean_plot, obs_plot],
    ["G", "G mean", "Observation"],
)

fig

Residual diagnostics

Plotting the ensemble tells you whether it is approaching the observation. analyze_residual tells you something more specific: how much of the remaining residual is structured, meaning aligned with the leading directions of the observational noise covariance, rather than noise-like.

import ClimaAnalysis   # required
result = ClimaCalibrate.analyze_residual(ekp, iteration; n_eigenvectors = 3)
result.structured_energy               # about 1 under the noise model
result.structured_energy_by_variable   # how that splits between variables
result.residual_norm_by_variable       # which variable dominates the residual

It projects obs - mean(G) onto the leading eigenvectors of the noise covariance, normalized by the corresponding eigenvalues, so the projections are z-scores: a value much larger than one means the residual has structure the noise model does not account for. A high structured energy in one variable points at that variable's observation or its part of the observation map.

This requires ClimaAnalysis to be loaded, and observations built by ObservationRecipe, since it uses their metadata to attribute the residual to individual variables. It supports SVD, Diagonal, and SVDplusD covariances.