NetCDFOutputWriter sometimes outputs twice at approximately the same time step
#3056
Comments
|
Sounds like a roundoff error. I've noticed similar problems. |
|
Maybe related to #2321 |
|
You're right, it does sound the same problem. My "extra time steps" also tend to happen at the end of long runs. |
|
MWE attached. This works for me on v0.91.0 and the issue seemingly doesn't require any TimeStepWizard. (It seems to take around 8 mins to run which feels slow given the problem size?) |
|
@tomchor, same results on the tc/timestepping branch |
Ah, bummer. I guess we need to investigate a little more. |
|
Can you copy paste the minimal code here? Hopefully it's sort (otherwise its' not minimal) |
using Oceananigans, Printf
Nt = 200 # number of time saves
T = 8e5*π/7 # simulation stop time (s)
Δt = 16/15 # timestep (s)
filename = "MWE_data" # save name of data file
architecture = CPU() # GPU() or CPU()
grid = RectilinearGrid(architecture, size = (), topology=(Flat, Flat, Flat))
model = NonhydrostaticModel(; grid, advection = CenteredFourthOrder(), timestepper = :RungeKutta3)
simulation = Simulation(model, Δt=Δt, stop_time=T)
progress_message(sim) = @printf("Iteration: %03d, time: %s, Δt: %s, wall time: %s\n",
iteration(sim), prettytime(sim), prettytime(sim.Δt), prettytime(sim.run_wall_time))
add_callback!(simulation, progress_message, IterationInterval(1000))
fields = Dict("u" => model.velocities.u);
simulation.output_writers[:field_writer] = NetCDFOutputWriter(model, fields,
filename = filename * ".nc", schedule = TimeInterval(T/Nt), overwrite_existing = true)
run!(simulation) |
|
Nice, thank you! I don't think it's necessary to use an output writer because schedules work the same for output and callbacks. Here's another possible MWE: using Oceananigans
Ns = 200 # number of time saves
T = 8e5*π/7 # simulation stop time (s)
Δt = 16/15 # timestep (s)
grid = RectilinearGrid(size = (), topology=(Flat, Flat, Flat))
model = NonhydrostaticModel(; grid)
simulation = Simulation(model; Δt, stop_time=T)
captured_times = []
capture_time(sim) = push!(captured_times, time(sim))
add_callback!(simulation, capture_time, TimeInterval(T/Ns))
run!(simulation)
@show captured_times |
|
Hmm, this actually doesn't reproduce the problem for me, although it reveals another problem wherein |
|
Here's an idea: rather than tracking |
|
Oh! maybe the time-stepper matters... |
|
Hmm, @tomchor had a notion for how to minimize the time-stepper effect too. |
|
Ok, updated MWE: using Oceananigans
FT = Float64
Ns = 200 # number of time saves
T = 8e5*π/7 # simulation stop time (s)
Δt = 16/15 # timestep (s)
grid = RectilinearGrid(FT, size = (), topology=(Flat, Flat, Flat))
model = NonhydrostaticModel(; grid, timestepper=:RungeKutta3)
simulation = Simulation(model; Δt, stop_time=T)
captured_times = []
capture_time(sim) = push!(captured_times, time(sim))
callback = Callback(capture_time, TimeInterval(T/Ns))
add_callback!(simulation, callback)
run!(simulation)
@show time(simulation) iteration(simulation)
@show length(captured_times)
@show time(simulation) == TI added an |
|
Ok yes with this change, I get 212 PS @mncrowe single backticks "`" format in-line code (not single quotes) and for code blocks you want to surround the block with triple backticks "```" (not double backticks) PPS compilation seems to stall for |
I noticed that occasionally the
NetCDFOutputWriterwill output twice at what is virtually the the same time.As an example, I'm showing below the time interval between time steps of one of my netcdf files created with a nominal
TimeIntervalof approximately 1236. However, you see that (while most of the times are separate by the correct amount) there are some instances with outputs separated by something much smaller:Furthermore, In this specific case, if the time steps were done correctly we would have 251 time steps, but we have 255.
Unfortunately I haven't been able to reproduce this in a MWE. My guess is that it needs a
TimeStepWizard(which I am using) and complex dynamics which make the wizard increase and decrease the time step relatively often (which also happens in my case).Two things to note:
The text was updated successfully, but these errors were encountered: