Private types and functions

Compute ZScore on time- and space-transformed field time series data.

compute_time_transformation(user_time_transformation, fts)

Compute a time transformation for the field time series fts given user_time_transformation.

By default, if user_time_transformation isa nothing, then we include all time instances except the initial condition.

FieldTimeSeriesCollector(collected_fields, times;
                         architecture = Architectures.architecture(first(collected_fields)))

Return a FieldTimeSeriesCollector for fields of simulation. fields is a NamedTuple of AbstractFields that are to be collected.

column_ensemble_interior(observations::Vector{<:SyntheticObservations},
                         field_name, time_index, (Nensemble, Nbatch, Nz))

Return an Nensemble × Nbatch × Nz Array of (1, 1, Nz) field_name data, given Nbatch SyntheticObservations objects. The Nbatch × Nz data for field_name is copied Nensemble times to form a 3D Array.

forward_map_names(observations::Vector{<:SyntheticObservations})

Return a Set representing the union of all names in obs.

observation_names(observations::Vector{<:SyntheticObservations})

Return a Set representing the union of all names in obs.

closure_with_parameters(closure, parameters)

Return a new object where for each (parameter_name, parameter_value) pair in parameters, the value corresponding to the key in closure that matches parameter_name is replaced with parameter_value.

Example

Create a placeholder Closure type that includes a parameter c and a sub-closure with two parameters: a and b. Then construct a closure with values a, b, c = 1, 2, 3.

julia> struct Closure; subclosure; c end

julia> struct ClosureSubModel; a; b end

julia> sub_closure = ClosureSubModel(1, 2)
ClosureSubModel(1, 2)

julia> closure = Closure(sub_closure, 3)
Closure(ClosureSubModel(1, 2), 3)

Providing closure_with_parameters with a named tuple of parameter names and values, and a recursive search in all types and subtypes within closure is done and whenever a parameter is found whose name exists in the named tuple we provided, its value is then replaced with the value provided.

julia> new_parameters = (a = 12, d = 7)
(a = 12, d = 7)

julia> using ParameterEstimocean.Parameters: closure_with_parameters

julia> closure_with_parameters(closure, new_parameters)
Closure(ClosureSubModel(12, 2), 3)

construct_object(specification_dict, parameters; name=nothing, type_parameter=nothing)

construct_object(d::ParameterValue, parameters; name=nothing)

Return a composite type object whose properties are prescribed by the specification_dict dictionary. All parameter values are given the values in specification_dict unless they are included as a parameter name-value pair in the named tuple parameters, in which case the value in parameters is asigned.

The construct_object is recursively called upon every property that is included in specification_dict until a property with a numerical value is reached. The object's constructor name must be included in specification_dict under key :type.

Example

julia> using ParameterEstimocean.Parameters: construct_object, dict_properties, closure_with_parameters

julia> struct Closure; subclosure; c end

julia> struct ClosureSubModel; a; b end

julia> sub_closure = ClosureSubModel(1, 2)
ClosureSubModel(1, 2)

julia> closure = Closure(sub_closure, 3)
Closure(ClosureSubModel(1, 2), 3)

julia> specification_dict = dict_properties(closure)
Dict{Symbol, Any} with 3 entries:
  :type       => Closure
  :c          => 3
  :subclosure => Dict{Symbol, Any}(:a=>1, :b=>2, :type=>ClosureSubModel)

julia> new_closure = construct_object(specification_dict, (a=2.1,))
Closure(ClosureSubModel(2.1, 2), 3)
  
julia> another_new_closure = construct_object(specification_dict, (b=π, c=2π))
Closure(ClosureSubModel(1, π), 6.283185307179586)

dict_properties(object)

Return a dictionary with all properties of an object and their values, including the object's type name. If any of the object's properties is not a numerical value but instead a composite type, then dict_properties is called recursively on that object's property returning a dictionary with all properties of that composite type. Recursion ends when properties of type ParameterValue are found.

new_closure_ensemble(closures, θ, arch=CPU())

Return a new set of closures in which all closures that have free parameters are updated. Closures with free parameters are expected as AbstractArray of TurbulenceClosures, and this allows new_closure_ensemble to go through all closures in closures and only update the parameters for the any closure that is of type AbstractArray. The architecture (CPU() or GPU()) defines whethere Array or CuArray is returned.

Return a logit-normally distributed variate given the normally-distributed variate X.

Return a normally-distributed variate given the logit-normally distributed variate Y.

transform_to_constrained(Π, X)

Transform an "unconstrained", normally-distributed variate X to "constrained" (physical) space via the map associated with the distribution Π of Y.

transform_to_unconstrained(Π, Y)

Transform the "constrained" (physical) variate Y into it's unconstrained (normally-distributed) counterpart X through the forward map associated with Π.

If some mapping between $Y$ and the normally-distributed $X$ is defined via

\[Y = g(X).\]

Then transform_to_unconstrained is the inverse $X = g^{-1}(Y)$. The change of variables $g(X)$ determines the distribution Π of Y.

Example

The logarithm of a LogNormal(μ, σ) distributed variate is normally-distributed, such that the forward trasform $f ≡ \exp$,

\[Y = \exp(X),\]

and the inverse trasnform is the natural logarithm $f^{-1} ≡ \log$,

\[\log(Y) = X ∼ 𝒩(μ, σ).\]

update_closure_ensemble_member!(closures, p_ensemble, parameters)

Use parameters to update the p_ensemble-th closure from and array of closures. The p_ensemble-th closure corresponds to ensemble member p_ensemble.

struct ConcatenatedVectorNormMap()

Forward map transformation of simulation output to a scalar by taking a naive norm of the difference between concatenated vectors of the observations and simulation output.

expand_parameters(ip, θ)

Convert θ to Vector{<:NamedTuple}, where the elements correspond to ip.free_parameters.

θ may represent an ensemble of parameter sets via:

• θ::Vector{<:Vector} (caution: parameters must be ordered correctly!)
• θ::Matrix (caution: parameters must be ordered correctly!)
• θ::Vector{<:NamedTuple}

or a single parameter set if θ::Vector{<:Number}.

If length(θ) is less the the number of ensemble members in ip.simulation, the last parameter set is copied to fill the parameter set ensemble.

inverting_forward_map(ip::InverseProblem, X)

Transform unconstrained parameters X into constrained, physical-space parameters θ and execute forward_map(ip, θ).

transform_time_series(map, batched_observations::Vector)

Concatenate the output of transform_time_series of each observation in batched_observations.

transform_time_series(::ConcatenatedOutputMap, observation::SyntheticObservations)

Transforms, normalizes, and concatenates data for field time series in observation.

transpose_model_output(time_series_collector, observations)

Transpose a NamedTuple of 4D FieldTimeSeries model output collected by time_series_collector into a Vector of SyntheticObservations for each member of the observation batch.

Return a 1-vector in the case of singleton observations.

IterationSummary(eki, X, forward_map_output=nothing)

Return the summary for ensemble Kalman inversion eki with unconstrained parameters X and forward_map_output.

Return a BitVector indicating which particles are NaN.

eki_objective(eki, θ, G)

Given forward map G and parameters θ, return a tuple (Φ₁, Φ₂) of terms in the EKI regularized objective function, where

Φ = Φ₁ + Φ₂

Φ₁ measures output misfit (1/2)*|| Γy^(-¹/₂) * (y .- G(θ)) ||² and Φ₂ measures prior misfit (1/2)*|| Γθ^(-¹/₂) * (θ .- μθ) ||², where y is the observation map, G(θ) is the forward map, Γy is the observation noise covariance, Γθ is the prior covariance, and μθ represents the prior means. Note that Γ^(-1/2) = inv(sqrt(Γ)). The keyword argument constrained is true if the input θ represents constrained parameters.

 find_successful_particles(eki, X, G, Nsample)

Generate Nsample new particles sampled from a multivariate Normal distribution parameterized by the ensemble mean and covariance computed based on the Nθ × Nensemble ensemble array θ, under the condition that all Nsample particles produce successful forward map outputs (don't include NaNs).

G (size(G) = Noutput × Nensemble) is the forward map output produced by θ.

Returns Nθ × Nsample parameter Array and Noutput × Nsample forward map output Array.

resample!(resampler::Resampler, θ, G, eki)

Resamples the parameters θ of the eki process based on the number of NaN values inside the forward map output G.