Scheduling callbacks and output writers

Callbacks and output writers in Simulation actuate on objects that are subtypes of AbstractSchedule. Schedules are small callable objects that return true when an action should fire and false otherwise. This page collects the built-in schedules, when to use them, and how to combine them for more complex behavior.

Aligned time steps

Simulation automatically shortens the next time step so that callback and output events scheduled by model time happen exactly when requested. Set align_time_step = false in the Simulation constructor to disable this.

For the examples here, we use the following simple simulation and progress:

using Oceananigans
grid = RectilinearGrid(size=(1, 1, 1), extent=(1, 1, 1))
model = NonhydrostaticModel(grid)
simulation = Simulation(model, Δt=0.1, stop_time=2.5, verbose=false)
dummy(sim) = @info string("Iter: ", iteration(sim), " -- I was called at t = ", time(sim),
                          " and wall time = ", prettytime(sim.run_wall_time))

dummy (generic function with 1 method)

During the examples below, we use Oceananigans.Simulations.reset!(simulation) so that we reuse the simulation in the next example without requiring to recreate it.

Basic schedules

IterationInterval

IterationInterval actuates every interval iterations:

schedule = IterationInterval(11)
add_callback!(simulation, dummy, schedule, name=:dummy)
run!(simulation)

[ Info: Iter: 0 -- I was called at t = 0.0 and wall time = 0 seconds
[ Info: Iter: 11 -- I was called at t = 1.0999999999999999 and wall time = 19.047 ms
[ Info: Iter: 22 -- I was called at t = 2.2000000000000006 and wall time = 19.165 ms

Use the offset keyword argument to shift the trigger so that, for example,

Oceananigans.Simulations.reset!(simulation)
simulation.stop_time = 2.5

schedule = IterationInterval(7; offset=-2)
add_callback!(simulation, dummy, schedule, name=:dummy)
run!(simulation)

[ Info: Iter: 0 -- I was called at t = 0.0 and wall time = 0 seconds
[ Info: Iter: 5 -- I was called at t = 0.5 and wall time = 256.404 μs
[ Info: Iter: 12 -- I was called at t = 1.2 and wall time = 334.131 μs
[ Info: Iter: 19 -- I was called at t = 1.9000000000000006 and wall time = 395.593 μs

Notice that other than iteration 0, the callback is actuated on iterations 5, 12, 19, ...

Above, we have overwritten the original callback called "dummy" with the new one with the offset schedule. An alternative way is to construct the Callback manually and add it in the simulation:

simulation.callbacks[:dummy] = Callback(dummy, schedule)

Callback of dummy on IterationInterval(7) with offset -2

TimeInterval

TimeInterval actuates every interval of model time, in units corresponding to model.clock.time. For example,

Oceananigans.Simulations.reset!(simulation)
simulation.stop_time = 2.5

schedule = TimeInterval(1.11)
add_callback!(simulation, dummy, schedule, name=:dummy)
run!(simulation)

[ Info: Iter: 0 -- I was called at t = 0.0 and wall time = 0 seconds
[ Info: Iter: 12 -- I was called at t = 1.11 and wall time = 16.791 ms
[ Info: Iter: 24 -- I was called at t = 2.22 and wall time = 17.022 ms

When model.clock.time isa AbstractTime such as DateTime, then interval can be Dates.Period:

using Dates

start_time = DateTime(2025, 1, 1)
clock = Clock(time = start_time)
datetime_model = NonhydrostaticModel(grid; clock)

stop_time = start_time + Dates.Minute(3)
datetime_simulation = Simulation(datetime_model; Δt=Dates.Second(25), stop_time, verbose=false)

schedule = TimeInterval(Dates.Minute(1))
add_callback!(datetime_simulation, dummy, schedule)
run!(datetime_simulation)

[ Info: Iter: 0 -- I was called at t = 2025-01-01T00:00:00 and wall time = 0 seconds
[ Info: Iter: 3 -- I was called at t = 2025-01-01T00:01:00 and wall time = 527.002 ms
[ Info: Iter: 6 -- I was called at t = 2025-01-01T00:02:00 and wall time = 527.209 ms
[ Info: Iter: 9 -- I was called at t = 2025-01-01T00:03:00 and wall time = 527.259 ms

WallTimeInterval

WallTimeInterval uses wall-clock seconds instead of model time. This is mostly useful for writing checkpoints to disk after consuming a fixed amount of computational resources. For example, using the previous simulation without the

Oceananigans.Simulations.reset!(simulation)
simulation.stop_time = 2.5

schedule = WallTimeInterval(0.005)
add_callback!(simulation, dummy, schedule, name=:dummy)
run!(simulation)

[ Info: Iter: 0 -- I was called at t = 0.0 and wall time = 0 seconds
[ Info: Iter: 1 -- I was called at t = 0.1 and wall time = 0 seconds

SpecifiedTimes

SpecifiedTimes actuates when model.clock.time reaches the given values. The constructor accepts numeric times or Dates.DateTime values and sorts them automatically. This schedule is helpful for pre-planned save points or events tied to specific model times.

Oceananigans.Simulations.reset!(simulation)
simulation.stop_time = 2.5

schedule = SpecifiedTimes(0.2, 1.5, 2.1)
add_callback!(simulation, dummy, schedule, name=:dummy)
run!(simulation)

[ Info: Iter: 0 -- I was called at t = 0.0 and wall time = 0 seconds
[ Info: Iter: 2 -- I was called at t = 0.2 and wall time = 22.293 ms
[ Info: Iter: 15 -- I was called at t = 1.5 and wall time = 22.538 ms
[ Info: Iter: 21 -- I was called at t = 2.1 and wall time = 22.603 ms

Arbitrary functions of model

Any function of model that returns a Bool can be used as a schedule:

Oceananigans.Simulations.reset!(simulation)
simulation.stop_time = 2.5

after_two(model) = model.clock.time > 2
add_callback!(simulation, dummy, after_two, name=:dummy)
run!(simulation)

[ Info: Iter: 0 -- I was called at t = 0.0 and wall time = 0 seconds
[ Info: Iter: 20 -- I was called at t = 2.0000000000000004 and wall time = 10.928 ms
[ Info: Iter: 21 -- I was called at t = 2.1000000000000005 and wall time = 11.031 ms
[ Info: Iter: 22 -- I was called at t = 2.2000000000000006 and wall time = 11.066 ms
[ Info: Iter: 23 -- I was called at t = 2.3000000000000007 and wall time = 11.095 ms
[ Info: Iter: 24 -- I was called at t = 2.400000000000001 and wall time = 11.117 ms
[ Info: Iter: 25 -- I was called at t = 2.5 and wall time = 11.139 ms

Combining schedules

Some applications benefit from running extra steps immediately after an event or from combining multiple criteria.

ConsecutiveIterations

ConsecutiveIterations actuates when the parent schedule does and for the next N iterations. For example, averaging callbacks often need data at the scheduled time and immediately afterwards.

Oceananigans.Simulations.reset!(simulation)
simulation.stop_time = 2.5

times = SpecifiedTimes(0.55, 1.5, 2.12)
schedule = ConsecutiveIterations(times)
add_callback!(simulation, dummy, schedule, name=:dummy)
run!(simulation)

[ Info: Iter: 0 -- I was called at t = 0.0 and wall time = 0 seconds
[ Info: Iter: 1 -- I was called at t = 0.1 and wall time = 0 seconds
[ Info: Iter: 6 -- I was called at t = 0.55 and wall time = 21.893 ms
[ Info: Iter: 7 -- I was called at t = 0.65 and wall time = 21.933 ms
[ Info: Iter: 16 -- I was called at t = 1.5 and wall time = 22.003 ms
[ Info: Iter: 17 -- I was called at t = 1.6 and wall time = 22.025 ms
[ Info: Iter: 23 -- I was called at t = 2.12 and wall time = 22.079 ms
[ Info: Iter: 24 -- I was called at t = 2.22 and wall time = 22.100 ms

TimeOffset

TimeOffset actuates when the parent schedule does and once more at a time offset away from it. A positive offset places the extra actuation after each parent actuation, and a negative offset places it before the next parent actuation. The time step is aligned so that the extra actuation lands exactly on the requested time. This is useful, for example, to sample a quantity a fixed time before each output is written, so that a time difference over that interval can be formed at writing time.

Oceananigans.Simulations.reset!(simulation)
simulation.stop_time = 2.5

schedule = TimeOffset(TimeInterval(1), -0.2)
add_callback!(simulation, dummy, schedule, name=:dummy)
run!(simulation)

[ Info: Iter: 0 -- I was called at t = 0.0 and wall time = 0 seconds
[ Info: Iter: 9 -- I was called at t = 0.8 and wall time = 29.364 ms
[ Info: Iter: 11 -- I was called at t = 1.0 and wall time = 29.515 ms
[ Info: Iter: 19 -- I was called at t = 1.8 and wall time = 29.586 ms
[ Info: Iter: 21 -- I was called at t = 2.0 and wall time = 29.618 ms

AndSchedule and OrSchedule

Use AndSchedule when an action should fire only if every child schedule actuates in the same iteration. Use OrSchedule when any one of the child schedules should trigger the action. Both accept any mix of AbstractSchedules, so you can require, for example, output every hour and every 1000 iterations:

Oceananigans.Simulations.reset!(simulation)
simulation.stop_time = 2.5

after_one_point_seven(model) = model.clock.time > 1.7
schedule = AndSchedule(IterationInterval(2), after_one_point_seven)
add_callback!(simulation, dummy, schedule, name=:dummy)
run!(simulation)

[ Info: Iter: 0 -- I was called at t = 0.0 and wall time = 0 seconds
[ Info: Iter: 18 -- I was called at t = 1.8000000000000005 and wall time = 36.223 ms
[ Info: Iter: 20 -- I was called at t = 2.0000000000000004 and wall time = 36.362 ms
[ Info: Iter: 22 -- I was called at t = 2.2000000000000006 and wall time = 36.401 ms
[ Info: Iter: 24 -- I was called at t = 2.400000000000001 and wall time = 36.435 ms

Stateful schedules

Stateful schedules such as TimeInterval, SpecifiedTimes, ConsecutiveIterations, and TimeOffset store their own counters, so we need to create a fresh instance (or call copy) for each callback or output writer that needs an identical pattern.

Output-specific schedules

Some schedules only apply to output writers because they keep extra state or require file access.

AveragedTimeInterval

AveragedTimeInterval asks an output writer to accumulate data over a sliding time window before writing. The window ends at each actuation time, runs for window seconds, and samples every stride iterations inside the window.

AveragedSpecifiedTimes

AveragedSpecifiedTimes behaves like SpecifiedTimes but with a trailing averaging window. Pass either a SpecifiedTimes instance or raw times.

FilteredTimeInterval

FilteredTimeInterval asks an output writer to write time-filtered output: a weighted average over the window of a kernel, centered on the output time, with weights from the kernel: LanczosKernel(window; cutoff), which removes periods shorter than cutoff, HanningKernel or BoxcarKernel (a running mean). Any AbstractFilterKernel can be supplied. Because the window is centered, the output carries no phase shift, unlike the trailing window of AveragedTimeInterval; each output is therefore written window / 2 after its time.

FileSizeLimit

FileSizeLimit actuates when the target file grows beyond size_limit bytes. Output writers update the internal path automatically so you usually only pass the size limit. Combine it with OrSchedule to rotate files when either the clock reaches a value or the file becomes too large.