Input/Output

HDF5Writer(filename::AbstractString,
           context::ClimaComms.AbstractCommsContext;
           overwrite::Bool = true)
HDF5Writer(::Function,
           filename::AbstractString,
           context::ClimaComms.AbstractCommsContext;
           overwrite::Bool = true)

Open filename for writing ClimaCore objects with the ClimaCore HDF5 storage conventions. Objects are written with write! and read back with HDF5Reader. The writer caches the domains, meshes, topologies, and grids it has written, so each is stored once per file.

Arguments

• filename: Path of the HDF5 file.
• context: The ClimaComms context of the run (ClimaComms.context()). For distributed fields it is the MPICommsContext the fields are distributed with; the file is then opened with MPI-IO.

Keyword Arguments

• overwrite = true: Replace an existing file. With overwrite = false, an existing file is opened for appending, and a missing file is created.

The do-block form passes the writer to the function and closes the file when the function returns. Both forms require context.

Examples

InputOutput.HDF5Writer(filename, ClimaComms.context()) do writer
    InputOutput.write!(writer, Y, "Y")
end

write!(writer::HDF5Writer, obj[, name])

Write a domain, mesh, topology, or grid obj to the file of writer and return the name it is stored under. name defaults to defaultname.

Each object is written once per file: writing an object that is already in the cache of writer leaves the file unchanged and returns the name it was first stored under. Distinct objects that request the same name (e.g. two spectral-element grids that differ only in a constructor flag, both named "horizontal_grid") are stored in distinct groups; the second and later ones get a _2, _3, ... suffix. References between objects use the returned name. Fields and FieldVectors are written with the three-argument method below, are not cached, and require an explicit name.

write!(writer::HDF5Writer, field::Fields.Field, name::AbstractString)
write!(writer::HDF5Writer, fieldvector::Fields.FieldVector, name::AbstractString)

Write field or fieldvector to the file of writer under name and return name.

A Field is stored as the dataset fields/<name>, with its data layout, element type, grid name, and staggering as attributes; the grid is written with write! if it is not in the file yet. A FieldVector is stored as the group fields/<name>, and each component is written as a Field (or a nested FieldVector) under <name>/<key>, so the component Y.c of a FieldVector named "Y" is stored as "Y/c". Fields and FieldVectors are not cached and can be written more than once under different names.

write!(
    writer::HDF5Writer,
    field::Fields.Field,
    name::AbstractString,
    space::Spaces.AbstractPointSpace,
)

Write a Field on a PointSpace to the file of writer. The field data is stored as the dataset fields/<name> and the local geometry data of the space as the dataset local_geometry_data/<name>, since a PointSpace has no grid to reference.

write!(
    writer::HDF5Writer,
    group,
    values::DataLayouts.DataLayout,
    name::AbstractString,
    topology::Topologies.AbstractTopology,
)

Write the DataLayout values as the dataset name in the HDF5 group. topology is the horizontal topology the data is laid out on; for a Topology2D with a distributed writer context, each rank writes its own elements with _write_mpi!, otherwise the whole array is written with _write!. Used for grid masks.

write!(
    writer::HDF5Writer,
    field::Fields.Field,
    name::AbstractString,
    space::Spaces.AbstractSpace,
)

Write a Field on space as the dataset fields/<name> and return name. The grid of space is written first with write!, and its name is stored in the grid attribute of the dataset, together with the data layout, element type, and staggering of the field. With a distributed writer context on a Topology2D, each rank writes its own elements with a collective MPI write.

write!(writer::HDF5Writer, name => value...)

Write one or more name => value pairs to writer, as write!(writer, value, name) for each pair. Return nothing.

write!(filename::AbstractString, name => value...)

Open an HDF5Writer on filename, write one or more name => value pairs to it, and close the file.

write_attributes!(writer::HDF5Writer, name::AbstractString, data::Dict)

Write the key-value pairs of data as attributes of the object at path name in the file of writer.

HDF5Reader(filename::AbstractString, context::ClimaComms.AbstractCommsContext)
HDF5Reader(::Function, filename::AbstractString, context::ClimaComms.AbstractCommsContext)

An AbstractReader for reading from HDF5 files created by HDF5Writer. The reader caches the domains, meshes, topologies, grids, and spaces it reads so that duplicate objects are not created.

context is the ClimaComms context of the run (ClimaComms.context()); with an MPICommsContext the resulting Fields are distributed over its ranks, which requires an HDF5 library with MPI support, as for HDF5Writer. The do-block form closes the file when the block returns. Opening a file written by a newer version of ClimaCore logs a warning.

Objects are read with read_domain, read_mesh, read_topology, read_space, and read_field.

Examples

InputOutput.HDF5Reader(filename, ClimaComms.context()) do reader
    Y = read_field(reader, "Y")          # the whole FieldVector
    Y.c |> propertynames
    Y.f |> propertynames
    c = read_field(reader, "Y/c")        # one component, by its slash path
    ρ_field = c.ρ
end

A FieldVector named "Y" with components c and f is stored as the fields "Y/c" and "Y/f"; the members of a NamedTuple-valued field (Y.c.ρ) are not addressable separately.

The caches expose what has been read so far, e.g. reader.space_cache is a Dict from space names such as "horizontal_space" to the space objects. Fields read this way can be plotted by loading a Makie backend (e.g. CairoMakie), which activates ClimaCore's plotting extension (ClimaCore.Visualize), and interpolated onto other grids with ClimaCoreTempestRemap.

read_domain(reader::AbstractReader, name)

Read the domain named name from reader, or from the reader cache if it has already been read.

read_mesh(reader::AbstractReader, name)

Read the mesh named name from reader, or from the reader cache if it has already been read.

read_topology(reader::AbstractReader, name)

Read the topology named name from reader, or from the reader cache if it has already been read. A Topology2D is created with the reader's context, so it is distributed when the context is an MPICommsContext.

read_space(reader::AbstractReader, name)

Read the space named name from reader, or from the reader cache if it has already been read.

read_field(reader, name)

Read the Field or FieldVector named name from reader. Fields are not cached, so reading the same field multiple times creates distinct objects. The components of a FieldVector are addressed by their slash path, e.g. "Y/c".

read_attributes(reader::AbstractReader, name::AbstractString)

Return a Dict of the attributes of the object at path name in the HDF5 file of reader.

defaultname(obj)

Return the default name under which write! stores a domain, mesh, topology, or grid, e.g. "sphere", "z-interval", "cubedsphere", or "horizontal_grid". Fields and FieldVectors have no default name.