Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add custom forcing velocity to particles #3396

Open
wants to merge 103 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 7 commits
Commits
Show all changes
103 commits
Select commit Hold shift + click to select a range
95f68a1
Fix particle-out-of-bounds problem
Yixiao-Zhang Oct 9, 2023
a0092d6
Add a comment
Yixiao-Zhang Oct 18, 2023
fc334a6
Merge branch 'main' into Yixiao-Zhang/fix-particles-out-of-bounds
Yixiao-Zhang Oct 18, 2023
29e5b07
Change architecture(model)
glwagner Oct 18, 2023
2143588
Edit comment
Yixiao-Zhang Oct 19, 2023
d02326d
Rewrite bounce_left and bounce_right more logically
Yixiao-Zhang Oct 19, 2023
3b6f574
Make LatitudeLongitudeGrid support Flat vertical directions
glwagner Oct 19, 2023
56cd1f9
Fix LatitutdeLongitudeGrid constructor for default topology
glwagner Oct 19, 2023
ed5c801
remove some stray spaces
navidcy Oct 20, 2023
06fbbd9
Merge branch 'main' into Yixiao-Zhang/fix-particles-out-of-bounds
glwagner Oct 25, 2023
64b409d
Refactor interpolate
glwagner Oct 30, 2023
0ce23f5
Eliminate merge artifacts
glwagner Oct 30, 2023
e2bb2cf
Bugfix
glwagner Oct 30, 2023
788d096
Merge branch 'main' into glw/better-simulation-interface
glwagner Oct 31, 2023
68406cd
Merge remote-tracking branch 'origin/glw/better-simulation-interface'…
glwagner Oct 31, 2023
8c4a063
Cosmetic improvement to WENO interpolants
glwagner Oct 31, 2023
6020ed1
Fix more interpolate bugs
glwagner Oct 31, 2023
e7c5ece
Refactor FieldTimeSeries a bit
glwagner Oct 31, 2023
3d8f8f9
Better input validation and show
glwagner Oct 31, 2023
3c691b3
More support for TotallyInMemory field time series and comment on 4D …
glwagner Nov 1, 2023
b615847
Fix error message
glwagner Nov 1, 2023
0420f55
Support AbstractTime for FieldTimeSeries
glwagner Nov 1, 2023
6ac1f39
Update src/Fields/interpolate.jl
glwagner Nov 1, 2023
0c8b340
Update src/Fields/interpolate.jl
glwagner Nov 1, 2023
c015b8b
Update src/Grids/latitude_longitude_grid.jl
glwagner Nov 1, 2023
f9e1093
Update src/Grids/latitude_longitude_grid.jl
glwagner Nov 1, 2023
c2b5d05
Tweak rectilinear grid constructor to match latitude longitude grid
glwagner Nov 1, 2023
a0f8fe2
Language tweak
glwagner Nov 1, 2023
901aa5f
Cosmetic updates
glwagner Nov 1, 2023
2e1080e
Merge remote-tracking branch 'origin/main' into glw/better-interpolate
glwagner Nov 1, 2023
e17c93e
Update manifest
glwagner Nov 2, 2023
0ae0bb4
Update data_dependencies.jl
simone-silvestri Nov 2, 2023
5dd7ed1
Add fill_halo_regions for FieldTimeSeries
glwagner Nov 2, 2023
4682818
Fix bug in time_indices
glwagner Nov 2, 2023
00a1627
Bump to 0.90.1
glwagner Nov 2, 2023
ca51fab
Update data_dependencies.jl
simone-silvestri Nov 2, 2023
a6bc090
Merge remote-tracking branch 'origin/ss/update-regression' into glw/b…
glwagner Nov 2, 2023
a054014
Update field interpolation tests
glwagner Nov 2, 2023
446f0e9
Bugfix
glwagner Nov 2, 2023
9055c28
Fix bug for NON flat grids
glwagner Nov 2, 2023
7df6229
Debug test_field
glwagner Nov 2, 2023
a867e55
Update Lagrangian particle tracking for new interpolate
glwagner Nov 2, 2023
48602bf
Bugfix
glwagner Nov 3, 2023
8ab3eef
Number is ok
glwagner Nov 3, 2023
32e0c98
Add utility for interpolating from one field to another
glwagner Nov 4, 2023
c21a5d7
Update Project.toml
glwagner Nov 6, 2023
cb39226
Merge branch 'main' into glw/better-interpolate
glwagner Nov 6, 2023
dd548a3
Revert Manifest
glwagner Nov 6, 2023
59b0aae
Fix puzzling bug in index_binary_search
glwagner Nov 10, 2023
49e05b8
Ok dont type restrict so much
glwagner Nov 10, 2023
e8a8224
Try a new way to interpolate
glwagner Nov 10, 2023
48ec4e6
Bugfix
glwagner Nov 10, 2023
21396d0
Did we fix interpolate tests?
glwagner Nov 10, 2023
83869ba
Fix bug in memory allocated field time series
glwagner Nov 10, 2023
8bff074
Merge branch 'main' into glw/better-interpolate
glwagner Nov 10, 2023
7ed8dc3
Fix bug in update_field_time_series!
glwagner Nov 10, 2023
c0f4037
Merge branch 'glw/better-interpolate' of https://github.com/CliMA/Oce…
glwagner Nov 10, 2023
46c411a
Fix problem for Flat lat lon grids
glwagner Nov 11, 2023
2d67753
Update field tests
glwagner Nov 14, 2023
778dd91
Put back allowscalar to get tests to pass
glwagner Nov 14, 2023
1841b2c
Expose all data not just interior
glwagner Nov 15, 2023
c7daaad
fix on interplate!
simone-silvestri Nov 17, 2023
0b6fcc8
first commit
simone-silvestri Nov 17, 2023
7f5ddf7
bugfix
simone-silvestri Nov 17, 2023
16c4816
Merge branch 'main' into glw/better-interpolate2
glwagner Nov 17, 2023
f40ab1f
Update lagrangian_particle_advection.jl
simone-silvestri Nov 20, 2023
a5aa48d
Update lagrangian_particle_advection.jl
simone-silvestri Nov 20, 2023
16f22bb
Update lagrangian_particle_advection.jl
simone-silvestri Nov 20, 2023
dd89374
Update lagrangian_particle_advection.jl
simone-silvestri Nov 20, 2023
370fab7
fix distributed tests
simone-silvestri Nov 20, 2023
c967882
fix distributed tests
simone-silvestri Nov 20, 2023
789d79e
bugfix
simone-silvestri Nov 20, 2023
2f4d470
Fix code re use problem in FieldTimeSeries
glwagner Nov 20, 2023
bf3b6f3
Merge branch 'glw/better-interpolate2' of https://github.com/CliMA/Oc…
glwagner Nov 20, 2023
dd384b6
Reshuffling and add interpolate! for FieldTimeSeries
glwagner Nov 21, 2023
dfdcf2c
No showing
glwagner Nov 21, 2023
a73e845
added ParticleAdvectionForcing
xkykai Dec 2, 2023
3a7f662
Merge remote-tracking branch 'origin/glw/better-interpolate2' into ss…
xkykai Dec 2, 2023
54ce501
not tested particle forcings interface
xkykai Dec 4, 2023
b53738a
Refactor particle advective forcing and velocities
xkykai Dec 28, 2023
959ddab
Fix order of operations in step_lagrangian_particles! function
xkykai Dec 28, 2023
b78c327
testing advective velocity with drag
xkykai Dec 28, 2023
e76406d
Update LagrangianParticleTracking.jl: Reorder code to compute dynamic…
xkykai Dec 29, 2023
b24448c
example code for particle with drag
xkykai Dec 29, 2023
c5540c0
cleanup advective forcing code
xkykai Dec 29, 2023
219e835
Update src/Models/LagrangianParticleTracking/lagrangian_particle_adve…
navidcy Dec 30, 2023
bfcfaa3
Merge branch 'main' into Yixiao-Zhang/fix-particles-out-of-bounds
navidcy Dec 30, 2023
8234689
Add buoyant particle validation test
xkykai Dec 30, 2023
96d970b
Add fluid velocity to advective velocity
xkykai Jan 1, 2024
aa7f47f
check bouncing in periodic domain
xkykai Jan 3, 2024
aa51f14
check left and right boundaries
xkykai Jan 3, 2024
bc9d6f3
Fix index calculation for periodic boundary conditions
xkykai Jan 3, 2024
fcdcd1b
Fix index calculation for periodic boundaries in lagrangian_particle_…
xkykai Jan 3, 2024
9a837da
Remove debug logging statements in lagrangian_particle_advection.jl
xkykai Jan 3, 2024
f88eb20
advect first then dynamics
xkykai Jan 3, 2024
06739c3
step particles before fields
xkykai Jan 3, 2024
fcd03f0
Merge branch 'main' into xk/bugfix-lagrangian-bugfix-rightmost-cell
navidcy Jan 8, 2024
edc3610
update to julia 1.10
xkykai Jan 17, 2024
fc7d129
Revert "update to julia 1.10"
xkykai Jan 17, 2024
b4eda3d
Merge branch 'main' into ss-xl/particle-advection
xkykai Feb 28, 2024
99a64f9
Merge branch 'xk/bugfix-lagrangian-bugfix-rightmost-cell' of https://…
xkykai Feb 28, 2024
d650c94
Merge remote-tracking branch 'yx/oceananigans/Yixiao-Zhang/fix-partic…
xkykai Mar 4, 2024
2cb0c3e
Merge branch 'main' into ss-xl/particle-advection
xkykai Jun 20, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
33 changes: 23 additions & 10 deletions src/Models/LagrangianParticleTracking/LagrangianParticleTracking.jl
Original file line number Diff line number Diff line change
Expand Up @@ -35,15 +35,17 @@ Base.show(io::IO, p::Particle) = print(io, "Particle at (",
@sprintf("%-8s", prettysummary(p.y, true) * ", "),
@sprintf("%-8s", prettysummary(p.z, true) * ")"))

struct LagrangianParticles{P, R, T, D, Π} <: AbstractLagrangianParticles
properties :: P
restitution :: R
tracked_fields :: T
dynamics :: D
parameters :: Π
struct LagrangianParticles{P, R, T, D, Π, F} <: AbstractLagrangianParticles
properties :: P
restitution :: R
tracked_fields :: T
dynamics :: D
parameters :: Π
advective_forcing :: F
xkykai marked this conversation as resolved.
Show resolved Hide resolved
end

@inline no_dynamics(args...) = nothing
@inline no_advective_forcing(args...) = nothing

"""
LagrangianParticles(; x, y, z, restitution=1.0, dynamics=no_dynamics, parameters=nothing)
Expand All @@ -54,7 +56,7 @@ Construct some `LagrangianParticles` that can be passed to a model. The particle
`dynamics` is a function of `(lagrangian_particles, model, Δt)` that is called prior to advecting particles.
`parameters` can be accessed inside the `dynamics` function.
"""
function LagrangianParticles(; x, y, z, restitution=1.0, dynamics=no_dynamics, parameters=nothing)
function LagrangianParticles(; x, y, z, restitution=1.0, dynamics=no_dynamics, parameters=nothing, advective_forcing=no_advective_forcing)
size(x) == size(y) == size(z) ||
throw(ArgumentError("x, y, z must all have the same size!"))

Expand All @@ -63,7 +65,7 @@ function LagrangianParticles(; x, y, z, restitution=1.0, dynamics=no_dynamics, p

particles = StructArray{Particle}((x, y, z))

return LagrangianParticles(particles; restitution, dynamics, parameters)
return LagrangianParticles(particles; restitution, dynamics, parameters, advective_forcing)
end

"""
Expand All @@ -83,15 +85,16 @@ function LagrangianParticles(particles::StructArray;
restitution = 1.0,
tracked_fields::NamedTuple=NamedTuple(),
dynamics = no_dynamics,
parameters = nothing)
parameters = nothing,
advective_forcing=no_advective_forcing)

for (field_name, tracked_field) in pairs(tracked_fields)
field_name in propertynames(particles) ||
throw(ArgumentError("$field_name is a tracked field but $(eltype(particles)) has no $field_name field! " *
"You might have to define your own particle type."))
end

return LagrangianParticles(particles, restitution, tracked_fields, dynamics, parameters)
return LagrangianParticles(particles, restitution, tracked_fields, dynamics, parameters, advective_forcing)
end

size(lagrangian_particles::LagrangianParticles) = size(lagrangian_particles.properties)
Expand All @@ -115,6 +118,16 @@ function Base.show(io::IO, lagrangian_particles::LagrangianParticles)
"└── dynamics: ", prettysummary(lagrangian_particles.dynamics, false))
end

struct ParticleAdvectiveForcing{U, V, W}
u :: U
v :: V
w :: W
end

function ParticleAdvectiveForcing(; u=no_advective_forcing, v=no_advective_forcing, w=no_advective_forcing)
return ParticleAdvectiveForcing(u, v, w)
end

include("update_lagrangian_particle_properties.jl")
include("lagrangian_particle_advection.jl")

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -37,21 +37,23 @@ const c = Center()
Return a new particle position if the position `(x, y, z)` lies in an immersed cell by
bouncing the particle off the immersed boundary with a coefficient or `restitution`.
"""
@inline function bounce_immersed_particle((x, y, z), grid, restitution, previous_particle_indices)
@inline function bounce_immersed_particle((x, y, z), ibg, restitution, previous_particle_indices)
X = flattened_node((x, y, z), ibg)

# Determine current particle cell
i, j, k = fractional_indices(x, y, z, (c, c, c), grid.underlying_grid)
i, j, k = fractional_indices(X, ibg.underlying_grid, (c, c, c))
i = Base.unsafe_trunc(Int, i)
j = Base.unsafe_trunc(Int, j)
k = Base.unsafe_trunc(Int, k)

if immersed_cell(i, j, k, grid)
if immersed_cell(i, j, k, ibg)
# Determine whether particle was _previously_ in a non-immersed cell
i⁻, j⁻, k⁻ = previous_particle_indices

if !immersed_cell(i⁻, j⁻, k⁻, grid)
if !immersed_cell(i⁻, j⁻, k⁻, ibg)
# Left-right bounds of the previous, non-immersed cell
xᴿ, yᴿ, zᴿ = node(i⁻+1, j⁻+1, k⁻+1, grid, f, f, f)
xᴸ, yᴸ, zᴸ = node(i⁻, j⁻, k⁻, grid, f, f, f)
xᴿ, yᴿ, zᴿ = node(i⁻+1, j⁻+1, k⁻+1, ibg, f, f, f)
xᴸ, yᴸ, zᴸ = node(i⁻, j⁻, k⁻, ibg, f, f, f)

Cʳ = restitution
x⁺ = enforce_boundary_conditions(Bounded(), x, xᴸ, xᴿ, Cʳ)
Expand All @@ -64,26 +66,49 @@ bouncing the particle off the immersed boundary with a coefficient or `restituti
return x⁺, y⁺, z⁺
end

"""
particle_u_velocity(u_fluid, particles, p, Δt)

a particle-specific advecting velocity such as
- sinking or rising for buoyant particles
- electromagnetic forces for charged particles
- swimming for biological particles
- diffusing velocity for brownian motion
Inputs are the fluid velocity, particle properties `particles`, and the particle index `p`

Returns the fluid velocity by default (for non-buoyant, non-drifting particles). Has to be extended to obtain the desired effect
"""
@inline particle_u_velocity(u_fluid, u_advective_forcing, particles, model, p, Δt) = u_fluid + u_advective_forcing(particles, model, Δt)
@inline particle_v_velocity(v_fluid, v_advective_forcing, particles, model, p, Δt) = v_fluid + v_advective_forcing(particles, model, Δt)
@inline particle_w_velocity(w_fluid, w_advective_forcing, particles, model, p, Δt) = w_fluid + w_advective_forcing(particles, model, Δt)

"""
advect_particle((x, y, z), p, restitution, grid, Δt, velocities)

Return new position `(x⁺, y⁺, z⁺)` for a particle at current position (x, y, z),
given `velocities`, time-step `Δt, and coefficient of `restitution`.
"""
@inline function advect_particle((x, y, z), p, restitution, grid, Δt, velocities)
@inline function advect_particle((x, y, z), p, particles, model, restitution, grid, Δt, velocities)
X = flattened_node((x, y, z), grid)

# Obtain current particle indices
i, j, k = fractional_indices(x, y, z, (c, c, c), grid)
i, j, k = fractional_indices(X, grid, c, c, c)
i = Base.unsafe_trunc(Int, i)
j = Base.unsafe_trunc(Int, j)
k = Base.unsafe_trunc(Int, k)

current_particle_indices = (i, j, k)

# Interpolate velocity to particle position
u = interpolate(velocities.u, f, c, c, grid, x, y, z)
v = interpolate(velocities.v, c, f, c, grid, x, y, z)
w = interpolate(velocities.w, c, c, f, grid, x, y, z)

u_fluid = interpolate(X, velocities.u, (f, c, c), grid)
v_fluid = interpolate(X, velocities.v, (c, f, c), grid)
w_fluid = interpolate(X, velocities.w, (c, c, f), grid)

# Particle velocity
u = particle_u_velocity(u_fluid, advective_forcing.u, particles, model, p, Δt)
v = particle_v_velocity(v_fluid, advective_forcing.v, particles, model, p, Δt)
w = particle_w_velocity(w_fluid, advective_forcing.w, particles, model, p, Δt)

# Advect particles, calculating the advection metric for a curvilinear grid.
# Note that all supported grids use length coordinates in the vertical, so we do not
# transform the vertical velocity nor invoke the k-index.
Expand Down Expand Up @@ -133,7 +158,7 @@ end
@inline y_metric(i, j, grid::RectilinearGrid) = 1
@inline y_metric(i, j, grid::LatitudeLongitudeGrid{FT}) where FT = 1 / grid.radius * FT(360 / 2π)

@kernel function _advect_particles!(particles, restitution, grid::AbstractUnderlyingGrid, Δt, velocities)
@kernel function _advect_particles!(particles, model, restitution, grid::AbstractUnderlyingGrid, Δt, velocities)
p = @index(Global)

@inbounds begin
Expand All @@ -142,7 +167,7 @@ end
z = particles.z[p]
end

x⁺, y⁺, z⁺ = advect_particle((x, y, z), p, restitution, grid, Δt, velocities)
x⁺, y⁺, z⁺ = advect_particle((x, y, z), p, particles, model, restitution, grid, Δt, velocities)

@inbounds begin
particles.x[p] = x⁺
Expand All @@ -158,8 +183,7 @@ function advect_lagrangian_particles!(particles, model, Δt)
worksize = length(particles)

advect_particles_kernel! = _advect_particles!(device(arch), workgroup, worksize)
advect_particles_kernel!(particles.properties, particles.restitution, model.grid, Δt, total_velocities(model))
advect_particles_kernel!(particles.properties, model, particles.restitution, model.grid, Δt, total_velocities(model))
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we cannot pass the model to a GPU kernel because we don't have an adapt function for the model.

You should extract the fields that you need and pass those to the kernel individually

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see. Is particles fine? What do you think the ParticleAdvectiveForcing arguments should contain? I'm imagining (particles, fluid_velocities, background_velocities, tracers, Δt) perhaps.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sounds good! maybe also auxiliary fields to allow customization?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

and the buoyancy model to allow for buoyant particles

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Put the same stuff in it that we put into the tendency kernel forcing functions.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is no i, j, k though because this is particles right? The equivalent is p, the particle index. So the discrete form arguments would be p, grid, clock, model_fields.

Shouldn't you multiply this by Δt (because its forward Euler forcing)? I don't think you need to pass in Δt.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The forcings are called here:

for example.

DiscreteForcing is a simple wrapper function that allows users to also store "parameters" in the forcing object.

Eventually, it makes sense to use the same here, and have "discreet form" and "continuous form" for ParticleForcing. It's up to you whether you want to go all the way. Those wrappers are pure convenience and do not add functionality.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

to be passed into ParticleAdvectionForcing

Should we call it simply ParticleForcing? It's true that this is a forcing on particle position and is therefore, in some sense, "advective". But I'm not sure it actually helps clarify how the forcing is used.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We would need i, j, k for model_fields unless we want to interpolate them and pass them

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, but what would i, j, k be? I guess to use model_fields, the user here would have to call interpolate manually, and they would also need to know the staggered location...

I do like the idea of interpolating the model fields, but to put that in we probably want to design an API where the interpolation only happens if the users want to use those fields, eg similar to how we have field_dependencies in ContinuousForcing. But as a first pass, we can simply avoid interpolating if isnothing(particle_forcing).


return nothing
end