Features

Abstract supertype for all random feature approximation models mapping inputs to random feature embeddings.

get_scalar_function(rf::RandomFeature) -> Any

gets the scalar_function field

get_feature_sampler(rf::RandomFeature) -> Any

gets the feature_sampler field

get_feature_sample(rf::RandomFeature) -> Any

gets the feature_sample field

get_n_features(rf::RandomFeature) -> Any

gets the n_features field

get_feature_parameters(rf::RandomFeature) -> Any

gets the feature_parameters field

get_output_dim(rf::ScalarFeature) -> Int64

gets the output dimension (equals 1 for scalar-valued features)

get_output_dim(rf::VectorFeature) -> Int64

Return the output dimension of rf.

sample(rf::RandomFeature) -> Any

samples the random feature distribution

struct ScalarFeature{S<:AbstractString, SF<:ScalarFunction} <: RandomFeature

Random feature model that maps N-dimensional inputs to 1-dimensional scalar outputs.

• n_features::Int64: Number of features

• feature_sampler::Sampler: Sampler of the feature distribution

• scalar_function::ScalarFunction: ScalarFunction mapping R -> R

• feature_sample::EnsembleKalmanProcesses.ParameterDistributions.ParameterDistribution: Current Sample from sampler

• feature_parameters::Union{Nothing, Dict{S}} where S<:AbstractString: hyperparameters in Feature (and not in Sampler)

Constructors

• ScalarFourierFeature(n_features, sampler; feature_parameters) — cosine (Fourier) features.
• ScalarNeuronFeature(n_features, sampler; activation_fun = Relu()) — activation-function features.

ScalarFourierFeature(
    n_features::Int64,
    sampler::Sampler;
    feature_parameters
) -> ScalarFeature{String, Cosine}

Construct a ScalarFeature with cosine (Fourier) features.

The sigma feature parameter (default sqrt(2)) scales the output; use larger values for wider kernel approximations.

ScalarNeuronFeature(
    n_features::Int64,
    sampler::Sampler;
    activation_fun,
    kwargs...
) -> ScalarFeature{String, Relu}

Construct a ScalarFeature with a neural-network activation function (default Relu).

Pass any ScalarActivation subtype as activation_fun to use a different activation.

build_features(
    rf::ScalarFeature,
    inputs::AbstractMatrix,
    batch_feature_idx::AbstractVector
) -> Any

Build random features from an input_dim × n_samples input matrix, evaluating only the feature indices in batch_feature_idx.

Returns an n_samples × 1 × length(batch_feature_idx) array.

struct VectorFeature{S<:AbstractString, SF<:ScalarFunction} <: RandomFeature

Random feature model that maps N-dimensional inputs to M-dimensional vector outputs.

• n_features::Int64: Number of features

• output_dim::Int64: Dimension of output

• feature_sampler::Sampler: Sampler of the feature distribution

• scalar_function::ScalarFunction: ScalarFunction mapping R -> R

• feature_sample::EnsembleKalmanProcesses.ParameterDistributions.ParameterDistribution: Current Sample from sampler

• feature_parameters::Union{Nothing, Dict{String}}: hyperparameters in Feature (and not in Sampler)

Constructors

• VectorFourierFeature(n_features, output_dim, sampler; feature_parameters) — cosine (Fourier) features.
• VectorNeuronFeature(n_features, output_dim, sampler; activation_fun = Relu()) — activation-function features.

VectorFourierFeature(
    n_features::Int64,
    output_dim::Int64,
    sampler::Sampler;
    feature_parameters
) -> VectorFeature{String, Cosine}

Construct a VectorFeature with cosine (Fourier) features mapping to output_dim-dimensional outputs.

The sigma feature parameter (default sqrt(2)) scales the output.

VectorNeuronFeature(
    n_features::Int64,
    output_dim::Int64,
    sampler::Sampler;
    activation_fun,
    kwargs...
) -> VectorFeature{String, Relu}

Construct a VectorFeature with a neural-network activation function (default Relu) mapping to output_dim-dimensional outputs.

Pass any ScalarActivation subtype as activation_fun to use a different activation.

build_features(
    rf::VectorFeature,
    inputs::AbstractMatrix,
    batch_feature_idx::AbstractVector
) -> Any

Build random features from an input_dim × n_samples input matrix, evaluating only the feature indices in batch_feature_idx.

Returns an n_samples × output_dim × length(batch_feature_idx) array.

abstract type ScalarFunction

Abstract supertype for all scalar functions mapping a single real value to a single real value.

abstract type ScalarActivation <: ScalarFunction

Abstract supertype for neural-network activation functions (a subtype of ScalarFunction).

apply_scalar_function(
    sf::ScalarFunction,
    r::AbstractArray
) -> Any

Apply the scalar function sf pointwise to each element of r.

Examples

julia> using RandomFeatures.Features

julia> apply_scalar_function(Relu(), [-1.0, 0.0, 1.0])
3-element Vector{Float64}:
 0.0
 0.0
 1.0

struct Cosine <: ScalarFunction

Cosine scalar function: applies cos(r) pointwise to each input.

struct Relu <: ScalarActivation

Rectified linear unit (ReLU) activation: returns max(0, r) for each input.

struct Lrelu{FT<:AbstractFloat} <: ScalarActivation

Leaky ReLU activation: linear for r > 0, scaled by alpha for r ≤ 0.

• alpha::AbstractFloat: Slope of the negative linear branch [dimensionless]; default 0.01.

Constructors

Lrelu(; alpha = 0.01)

struct Gelu <: ScalarActivation

Gaussian error linear unit (GELU) activation: r * Φ(r), where Φ is the standard normal CDF.

struct Elu{FT<:AbstractFloat} <: ScalarActivation

Exponential linear unit (ELU) activation: linear for r > 0, scaled exponential alpha * (exp(r) - 1) for r ≤ 0.

• alpha::AbstractFloat: Scale applied to the negative exponential branch [dimensionless]; default 1.0.

Constructors

Elu(; alpha = 1.0)

struct Selu{FT<:AbstractFloat} <: ScalarActivation

Scaled ELU (SELU) activation: self-normalising variant of ELU with fixed default scale parameters.

• alpha::AbstractFloat: Negative-branch scale factor for self-normalisation [dimensionless]; default 1.67326.

• lambda::AbstractFloat: Global scale factor for self-normalisation [dimensionless]; default 1.0507.

Constructors

Selu(; alpha = 1.67326, lambda = 1.0507)

struct Heaviside <: ScalarActivation

Heaviside step function: returns 0 for r < 0, 0.5 at r == 0, and 1 for r > 0.

struct SmoothHeaviside{FT<:AbstractFloat} <: ScalarActivation

Smooth differentiable approximation to the Heaviside step function: 0.5 + atan(r / epsilon) / π.

• epsilon::AbstractFloat: Width of the transition region [dimensionless]; smaller values approach the hard step function. Default 0.01.

Constructors

SmoothHeaviside(; epsilon = 0.01)

struct Sawtooth <: ScalarActivation

Sawtooth (triangle wave) activation: max(0, min(2r, 2 - 2r)).

struct Softplus <: ScalarActivation

Softplus activation: numerically stable smooth approximation to ReLU, computed as log(1 + exp(-|r|)) + max(r, 0).

struct Tansig <: ScalarActivation

Hyperbolic tangent sigmoid activation: returns tanh(r).

struct Sigmoid <: ScalarActivation

Logistic sigmoid activation: returns 1 / (1 + exp(-r)).