Observations

Observation

EnsembleKalmanProcesses.ObservationType
Observation

Structure that contains a (possibly stacked) observation. Defined by sample(s), noise covariance(s), and name(s)

Typical Constructors:

Observation(
    Dict(
        "samples" => [1,2,3],
        "covariances" => I(3),
        "names" => "one_two_three",
    ),
)

or

Observation([1,2,3], I(3), "one_two_three")

One can stack up multiple observations with combine_observations, or by providing vectors of samples, covariances and names to the dictionary.

Fields

  • samples::AbstractVector

    A (vector of) observation vectors

  • covs::AbstractVector

    A (vector of) observation covariance matrices

  • inv_covs::AbstractVector

    A (vector of) inverses of observation covariance matrices

  • names::AbstractVector

    A (vector of) name strings

  • indices::AbstractVector

    A (vector of) indices of the contained observation blocks

source
EnsembleKalmanProcesses.combine_observationsFunction
combine_observations(obs_vec::AbstractVector) -> Observation{AV1, AV2, AV3, AV4, AV5} where {AV1<:(Vector), AV2<:(Vector), AV3<:(Vector), AV4<:(Vector), AV5<:(Vector)}

combines a vector of Observation objects into a single Observation

source
EnsembleKalmanProcesses.get_obsMethod
get_obs(o::Observation) -> Any

if build=true, returns the stacked vector of observed samples samples(default), otherwise it calls get_samples

source

Covariance utilities

EnsembleKalmanProcesses.tsvd_matFunction
tsvd_mat(X, r::Int64) -> Any

For a given matrix X and rank r, return the truncated SVD for X as a LinearAlgebra.jl SVD object. Setting return_inverse=true also return it's psuedoinverse X⁺.

source
EnsembleKalmanProcesses.tsvd_cov_from_samplesFunction
tsvd_cov_from_samples(sample_mat::AbstractMatrix) -> Union{Tuple{LinearAlgebra.SVD, LinearAlgebra.SVD}, LinearAlgebra.SVD}

For a given sample_mat, (with data_are_columns = true), rank "r" is optionally provided. Returns the SVDs corresponding to the matrix cov(sample_mat; dims=2). Efficient representation when size(samplemat,1) << size(samplemat,2). Setting return_inverse=true also returns its psuedoinverse.

source
EnsembleKalmanProcesses.SVDplusDType
SVDplusD

Storage for a covariance matrix of the form D + USV' for Diagonal D, and SVD decomposition USV'. Note the inverse of this type (as computed through inv_cov(...)) will be stored compactly as a DminusTall type.

  • svd_cov::LinearAlgebra.SVD

    summand of covariance matrix stored with SVD decomposition

  • diag_cov::LinearAlgebra.Diagonal

    summand of covariance matrix stored as a diagonal matrix

source
EnsembleKalmanProcesses.DminusTallType
DminusTall

Storage for a covariance matrix of the form D - RR' for Diagonal D, and (tall) matrix R. Primary use case for this matrix is to compactly store the inverse of the SVDplusD type.

  • diag_cov::LinearAlgebra.Diagonal

    summand of covariance matrix stored as a diagonal matrix

  • tall_cov::AbstractMatrix

    summand of covariance matrix stored as an abstract matrix

source

Minibatcher

EnsembleKalmanProcesses.FixedMinibatcherType
FixedMinibatcher <: Minibatcher

A Minibatcher that takes in a given epoch of batches. It creates a new epoch by either copying-in-order, or by shuffling, the provided batches.

Fields

  • minibatches::AbstractVector

    explicit indices of the minibatched epoch

  • method::AbstractString

    method of selecting minibatches from the list for each epoch ("order" select in order, "random" generate a random selector)

  • rng::Random.AbstractRNG

    rng for sampling, if "random" method is selected

source
EnsembleKalmanProcesses.no_minibatcherFunction
no_minibatcher() -> FixedMinibatcher{Vector{T}, String, Random.TaskLocalRNG} where T<:(AbstractVector)
no_minibatcher(epoch_size::Int64) -> FixedMinibatcher{Vector{T}, String, Random.TaskLocalRNG} where T<:(AbstractVector)

constructs a FixedMinibatcher of given epoch_size, that generates an epoch of 1:epoch_size and one minibatch that constitutes the whole epoch

source
EnsembleKalmanProcesses.RandomFixedSizeMinibatcherType
RandomFixedSizeMinibatcher <: Minibatcher

A Minibatcher that takes in a given epoch of batches. It creates a new epoch by either copying-in-order, or by shuffling, the provided batches.

Fields

  • minibatch_size::Int64

    fixed size of minibatches

  • method::AbstractString

    how to deal with remainder if minibatch-size doesn't divide the epoch size ("trim" - ignore trailing samples, "extend" - have a larger final minibatch)

  • rng::Random.AbstractRNG

    rng for sampling

  • minibatches::AbstractVector

    explicit indices of the minibatched epoch

source

ObservationSeries

EnsembleKalmanProcesses.ObservationSeriesType
ObservationSeries

Structure that contains multiple Observations along with an optional Minibatcher. Stores all observations in EnsembleKalmanProcess, as well as defining the behavior of the get_obs, get_obs_noise_cov, and get_obs_noise_cov_inv methods

Typical Constructor

ObservationSeries(
    Dict(
        "observations" => vec_of_observations,
        "names" => names_of_observations,
        "minibatcher" => minibatcher,
    ),
)

Fields

  • observations::AbstractVector

    A vector of Observations to be used in the experiment

  • minibatcher::Minibatcher

    A Minibatcher object used to define the minibatching

  • names::AbstractVector

    A vector of string identifiers for the observations

  • current_minibatch_index::Dict

    The current index (epoch #, minibatch #) of the current minibatch, stored as a Dict

  • minibatches::AbstractVector

    The batch history (grouped by minibatch and epoch)

  • metadata::Any

    Metadata of any type that the user can group with the ObservationSeries

source
EnsembleKalmanProcesses.update_minibatch!Method

Within an epoch: iterates the current minibatch index by one. At the end of an epoch: obtains a new epoch of minibatches from the Minibatcher updates the epoch index by one, and minibatch index to one.

source
EnsembleKalmanProcesses.get_minibatchFunction
get_minibatch(os::ObservationSeries, it_or_mbi::Union{Nothing, Int64, Dict}) -> Any

get the minibatch for a given minibatch index (Dict("epoch"=> x, "minibatch" => y)), or iteration Int. If nothing is provided as an iteration then the current minibatch is returned

source
EnsembleKalmanProcesses.get_obsMethod
get_obs(os::ObservationSeries) -> Any

if build=true then gets the observed sample, stacked over the current minibatch. build=false lists the samples for all observations.

source
EnsembleKalmanProcesses.get_obsMethod
get_obs(os::Union{Nothing, Int64})

if build=true then gets the observed sample, stacked over the minibatch at iteration. build=false lists the samples for all observations. If isnothing(iteration) or not defined then the current iteration is used.

source
EnsembleKalmanProcesses.get_obs_noise_covMethod

if build=true then gets the observation covariance matrix, blocked over the minibatch at iteration. build=false lists the covs for all observations. If isnothing(iteration) or not defined then the current iteration is used.

source
EnsembleKalmanProcesses.get_obs_noise_cov_invMethod

if build=true then gets the inverse of the observation covariance matrix, blocked over minibatch at iteration. build=false lists the inv_covs for all observations. If isnothing(iteration) or not defined then the current iteration is used.

source