How-to guide and cookbook

How do I reconstruct the observations as OutputVars?

If you are using ObservationRecipe, you can reconstruct an EKP.Observation with ObservationRecipe.reconstruct_vars.

import ClimaCalibrate: ObservationRecipe
import ClimaAnalysis

# obs is a `EKP.Observation`
# Return a vector of OutputVar
vars = ObservationRecipe.reconstruct_vars(obs)

# obs_series is a `EKP.ObservationSeries`
# Return a vector of OutputVars reconstructed from the first observation in the
# series
vars =
    obs_series |>
    EKP.get_observations |>
    first |>
    ObservationRecipe.reconstruct_vars

# ekp is an `EKP.EnsembleKalmanProcess` object
# Same result as the example above
vars =
    ekp |>
    EKP.get_observation_series |>
    EKP.get_observations |>
    first |>
    ObservationRecipe.reconstruct_vars

How do I reconstruct the G ensemble matrix as OutputVars?

If you are using ObservationRecipe, you can reconstruct the G ensemble matrix as OutputVars with ObservationRecipe.reconstruct_g, ObservationRecipe.reconstruct_g_mean, and ObservationRecipe.reconstruct_g_mean_final.

import ClimaCalibrate: ObservationRecipe
import ClimaAnalysis

# ekp is an `EKP.EnsembleKalmanProcess` object

# Return a matrix of `OutputVar`s reconstructed from the G ensemble matrix
# at the 2nd iteration
G_ens_mat_as_vars = ObservationRecipe.reconstruct_g(ekp, 2)

# Return a vector of `OutputVar` reconstructed from the mean forward map
# evaluation at the 3rd iteration
g_mean_as_vars = ObservationRecipe.reconstruct_g_mean(ekp, 3)

# Return a vector of `OutputVar` reconstructed from the final mean forward map
# evaluation at the final iteration
ObservationRecipe.reconstruct_g_mean_final(ekp)

I have a diagonal covariance matrix. How can I reconstruct this as a OutputVar?

If you created the diagonal covariance matrix with ObservationRecipe, you can reconstruct with ObservationRecipe.reconstruct_diag_cov where obs is a EKP.Observation.

import ClimaCalibrate: ObservationRecipe
import ClimaAnalysis

ObservationRecipe.reconstruct_diag_cov(obs)

How do I handle NaNs in the OutputVars so that there are no NaNs in the sample and covariance matrix?

NaNs should be handled when preprocessing the data. In some cases, there will be NaNs in the data (e.g. calibrating with data that is valid only over land). In these cases, the SampleBuilder module automatically removes NaNs from the data when building the samples, since ClimaAnalysis.flatten drops NaNs while flattening each OutputVar. It is important to ensure that across the time slices, the NaNs appear in the same coordinates of the non-temporal dimensions. For example, if the quantity is defined over the dimensions longitude, latitude, and time, then any slice of the data at a particular longitude and latitude should either only contain NaNs or no NaNs at all. The SampleBuilder module checks that the dropped coordinates match across samples and errors otherwise.