DSS

dss_transform(arg, local_geometry, weight, I)

Transfrom arg[I] to a basis for direct stiffness summation (DSS). Transformations only apply to vector quantities.

• local_geometry[I] is the relevant LocalGeometry object. If it is nothing, then no transformation is performed
• weight[I] is the relevant DSS weights. If weight is nothing, then the result is simply summation.

See ClimaCore.Spaces.weighted_dss!.

dss_transform!(
    device::ClimaComms.AbstractDevice,
    dss_buffer::DSSBuffer,
    data::Union{DataLayouts.IJFH, DataLayouts.IJHF, DataLayouts.VIJFH, DataLayouts.VIJHF},
    local_geometry::Union{DataLayouts.IJFH, DataLayouts.IJHF, DataLayouts.VIJFH, DataLayouts.VIJHF},
    dss_weights::Union{DataLayouts.IJFH, DataLayouts.IJHF, DataLayouts.VIJFH, DataLayouts.VIJHF},
    perimeter::Perimeter2D,
    localelems::AbstractVector{Int},
)

Transforms vectors from Covariant axes to physical (local axis), weights the data at perimeter nodes, and stores result in the perimeter_data array. This function calls the appropriate version of dss_transform! based on the data layout of the input arguments.

Arguments:

• dss_buffer: DSSBuffer generated by create_dss_buffer function for field data
• data: field data
• local_geometry: local metric information defined at each node
• dss_weights: local dss weights for horizontal space
• perimeter: perimeter iterator
• localelems: list of local elements to perform transformation operations on

Part of ClimaCore.Spaces.weighted_dss!.

function dss_transform!(
    ::ClimaComms.AbstractCPUDevice,
    perimeter_data::Union{DataLayouts.VIFH, DataLayouts.VIHF},
    data::Union{DataLayouts.IJFH, DataLayouts.IJHF, DataLayouts.VIJFH, DataLayouts.VIJHF},
    perimeter::Perimeter2D,
    local_geometry::Union{DataLayouts.IJFH, DataLayouts.IJHF, DataLayouts.VIJFH, DataLayouts.VIJHF},
    dss_weights::Union{DataLayouts.IJFH, DataLayouts.IJHF, DataLayouts.VIJFH, DataLayouts.VIJHF},
    localelems::Vector{Int},
)

Transforms vectors from Covariant axes to physical (local axis), weights the data at perimeter nodes, and stores result in the perimeter_data array.

Arguments:

• perimeter_data: contains the perimeter field data, represented on the physical axis, corresponding to the full field data in data
• data: field data
• perimeter: perimeter iterator
• local_geometry: local metric information defined at each node
• dss_weights: local dss weights for horizontal space
• localelems: list of local elements to perform transformation operations on

Part of ClimaCore.Spaces.weighted_dss!.

dss_untransform!(
    device::ClimaComms.AbstractDevice,
    dss_buffer::DSSBuffer,
    data::Union{DataLayouts.IJFH, DataLayouts.IJHF, DataLayouts.VIJFH, DataLayouts.VIJHF},
    local_geometry::Union{DataLayouts.IJFH, DataLayouts.IJHF, DataLayouts.VIJFH, DataLayouts.VIJHF},
    perimeter::AbstractPerimeter,
)

Transforms the DSS'd local vectors back to Covariant12 vectors, and copies the DSS'd data from the perimeter_data to data. This function calls the appropriate version of dss_transform! function based on the data layout of the input arguments.

Arguments:

• dss_buffer: DSSBuffer generated by create_dss_buffer function for field data
• data: field data
• local_geometry: local metric information defined at each node
• perimeter: perimeter iterator
• localelems: list of local elements to perform transformation operations on

Part of ClimaCore.Spaces.weighted_dss!.

dss_untransform(T, targ, local_geometry, I...)

Transform targ[I...] back to a value of type T after performing direct stiffness summation (DSS).

See ClimaCore.Spaces.weighted_dss!.

dss_local_vertices!(
    perimeter_data::DataLayouts.VIFH,
    perimeter::Perimeter2D,
    topology::Topology2D,
)

Apply dss to local vertices.

function dss_local!(
    ::ClimaComms.AbstractCPUDevice,
    perimeter_data::DataLayouts.VIFH,
    perimeter::AbstractPerimeter,
    topology::AbstractTopology,
)

Performs DSS on local vertices and faces.

Part of ClimaCore.Spaces.weighted_dss!.

function dss_local_ghost!(
    ::ClimaComms.AbstractCPUDevice,
    perimeter_data::DataLayouts.VIFH,
    perimeter::AbstractPerimeter,
    topology::AbstractTopology,
)

Computes the "local" part of ghost vertex dss. (i.e. it computes the summation of all the shared local vertices of a unique ghost vertex and stores the value in each of the local vertex locations in perimeter_data)

Part of ClimaCore.Spaces.weighted_dss!.

dss_ghost!(
    device::ClimaComms.AbstractCPUDevice,
    perimeter_data::DataLayouts.VIFH,
    perimeter::AbstractPerimeter,
    topology::AbstractTopology,
)

Sets the value for all local vertices of each unique ghost vertex, in perimeter_data, to that of the representative ghost vertex.

Part of ClimaCore.Spaces.weighted_dss!.

create_dss_buffer(data, space)

Creates a DSSBuffer for the field data corresponding to data

Spaces.create_dss_buffer(fv::FieldVector)

Create a NamedTuple of buffers for communicating neighbour information of each Field in fv. In this NamedTuple, the name of each field is mapped to the buffer.

Spaces.create_dss_buffer(field::Field)

Create a buffer for communicating neighbour information of field.

create_dss_buffer(
    data::Union{DataLayouts.IJFH{S}, DataLayouts.IJHF{S}, DataLayouts.VIJFH{S}, DataLayouts.VIJHF{S}},
    topology::Topology2D,
    local_geometry::Union{DataLayouts.IJFH, DataLayouts.IJHF, DataLayouts.VIJFH, DataLayouts.VIJHF, Nothing} = nothing,
    dss_weights::Union{DataLayouts.IJFH, DataLayouts.IJHF, DataLayouts.VIJFH, DataLayouts.VIJHF, Nothing} = nothing,
) where {S}

Creates a DSSBuffer for the field data corresponding to data

fill_send_buffer!(::ClimaComms.AbstractCPUDevice, dss_buffer::DSSBuffer; synchronize=true)

Loads the send buffer from perimeter_data. For unique ghost vertices, only data from the representative vertices which store result of "ghost local" DSS are loaded.

Part of ClimaCore.Spaces.weighted_dss!.

DSSBuffer{G, D, A, B}

Fields

• graph_context: ClimaComms graph context for communication
• perimeter_data: Perimeter DataLayout object: typically a VIFH{TT,Nv,Np,Nh} or VIHF{TT,Nv,Np,Nh}, where TT is the transformed type, Nv is the number of vertical levels, and Np is the length of the perimeter
• send_date: send buffer AbstractVector{FT}
• recv_data: recv buffer AbstractVector{FT}
• send_buf_idx: indexing array for loading send buffer from perimeter_data
• recv_buf_idx: indexing array for loading (and summing) data from recv buffer to
• internal_elems: internal local elements (lidx)
• perimeter_elems: local elements (lidx) located on process boundary

load_from_recv_buffer!(::ClimaComms.AbstractCPUDevice, dss_buffer::DSSBuffer)

Adds data from the recv buffer to the corresponding location in perimeter_data. For ghost vertices, this data is added only to the representative vertices. The values are then scattered to other local vertices corresponding to each unique ghost vertex in dss_local_ghost.

Part of ClimaCore.Spaces.weighted_dss!.

dss!(data, topology)

Computed unweighted/pure DSS of data.

weighted_dss_start!(
    data::Union{
        DataLayouts.IFH,
        DataLayouts.IHF,
        DataLayouts.VIFH,
        DataLayouts.VIHF,
        DataLayouts.IJFH,
        DataLayouts.IJHF,
        DataLayouts.VIJFH,
        DataLayouts.VIJHF,
    },
    space::Union{
        AbstractSpectralElementSpace,
        ExtrudedFiniteDifferenceSpace,
    },
    dss_buffer::Union{DSSBuffer, Nothing},
)

It comprises of the following steps:

1). Apply Spaces.dss_transform! on perimeter elements. This weights and tranforms vector fields to physical basis if needed. Scalar fields are weighted. The transformed and/or weighted perimeter data is stored in perimeter_data.

2). Apply Spaces.dss_local_ghost! This computes partial weighted DSS on ghost vertices, using only the information from local vertices.

3). Spaces.fill_send_buffer! Loads the send buffer from perimeter_data. For unique ghost vertices, only data from the representative ghost vertices which store result of "ghost local" DSS are loaded.

4). Start DSS communication with neighboring processes

weighted_dss_internal!(
    data::Union{
        DataLayouts.IFH,
        DataLayouts.IHF,
        DataLayouts.VIFH,
        DataLayouts.VIHF,
        DataLayouts.IJFH,
        DataLayouts.IJHF,
        DataLayouts.VIJFH,
        DataLayouts.VIJHF,
    },
    space::Union{
        AbstractSpectralElementSpace,
        ExtrudedFiniteDifferenceSpace,
    },
    dss_buffer::DSSBuffer,
)

1). Apply Spaces.dss_transform! on interior elements. Local elements are split into interior and perimeter elements to facilitate overlapping of communication with computation.

2). Probe communication

3). Spaces.dss_local! computes the weighted DSS on local vertices and faces.

weighted_dss_ghost!(
    data::Union{
        DataLayouts.IFH,
        DataLayouts.IHF,
        DataLayouts.VIFH,
        DataLayouts.VIHF,
        DataLayouts.IJFH,
        DataLayouts.IJHF,
        DataLayouts.VIJFH,
        DataLayouts.VIJHF,
    },
    space::Union{
        AbstractSpectralElementSpace,
        ExtrudedFiniteDifferenceSpace,
    },
    dss_buffer::Union{DSSBuffer, Nothing},
)

1). Finish communications.

2). Call Spaces.load_from_recv_buffer! After the communication is complete, this adds data from the recv buffer to the corresponding location in perimeter_data. For ghost vertices, this data is added only to the representative vertices. The values are then scattered to other local vertices corresponding to each unique ghost vertex in dss_local_ghost.

3). Call Spaces.dss_untransform! on all local elements. This transforms the DSS'd local vectors back to Covariant12 vectors, and copies the DSS'd data from the perimeter_data to data.

function weighted_dss!(
    data::Union{
        DataLayouts.IFH,
        DataLayouts.IHF,
        DataLayouts.VIFH,
        DataLayouts.VIHF,
        DataLayouts.IJFH,
        DataLayouts.IJHF,
        DataLayouts.VIJFH,
        DataLayouts.VIJHF,
    },
    space::Union{
        AbstractSpectralElementSpace,
        ExtrudedFiniteDifferenceSpace,
    },
    dss_buffer::Union{DSSBuffer, Nothing},
)

Computes weighted dss of data.

It comprises of the following steps:

1). Spaces.weighted_dss_start!

2). Spaces.weighted_dss_internal!

3). Spaces.weighted_dss_ghost!

Spaces.weighted_dss!(fv::FieldVector, dss_buffer = Spaces.create_dss_buffer(fv))

Apply weighted direct stiffness summation (DSS) to each field in fv. If a dss_buffer object is not provided, a buffer will be created for each field in fv. Note that using the Pair interface here parallelizes the weighted_dss! calls.

Spaces.weighted_dss!(f::Field, dss_buffer = Spaces.create_dss_buffer(field))

Apply weighted direct stiffness summation (DSS) to f. This operates in-place (i.e. it modifies the f). ghost_buffer contains the necessary information for communication in a distributed setting, see Spaces.create_dss_buffer.

This is a projection operation from the piecewise polynomial space $\mathcal{V}_0$ to the continuous space $\mathcal{V}_1 = \mathcal{V}_0 \cap \mathcal{C}_0$, defined as the field $\theta \in \mathcal{V}_1$ such that for all $\phi \in \mathcal{V}_1$

\[\int_\Omega \phi \theta \,d\Omega = \int_\Omega \phi f \,d\Omega\]

In matrix form, we define $\bar \theta$ to be the unique global node representation, and $Q$ to be the "scatter" operator which maps to the redundant node representation $\theta$

\[\theta = Q \bar \theta\]

Then the problem can be written as

\[(Q \bar\phi)^\top W J Q \bar\theta = (Q \bar\phi)^\top W J f\]

which reduces to

\[\theta = Q \bar\theta = Q (Q^\top W J Q)^{-1} Q^\top W J f\]

Spaces.weighted_dss!(field1 => ghost_buffer1, field2 => ghost_buffer2, ...)

Call Spaces.weighted_dss! on multiple fields at once, overlapping communication as much as possible.

unique_nodes(space::SpectralElementSpace2D)

An iterator over the unique nodes of space. Each node is represented by the first ((i,j), e) triple.

This function is experimental, and may change in future.