Is it possible to constrain tracking by max velocity?

[aperkes]

Right now I'm tracking 4 fish in a long-term tank which is divided into partitions. Because fish are staying in for weeks at a time, we need a standpipe, so the fish frequently vanish behind that.

For the most part, tracking works well, but often we get switched tracks when one fish is lost. (it's even worse when another fish appears at the same time). I understand that this is a bit of a tricky matching problem, but in our case it would be pretty easy to solve this problem if tracking had a max-velocity parameter that did not allow teleportation. For this specific use case, it's pretty easy to catch and correct this, or just workaround it, since we have defined regions where the fish can be, but we'll soon be using the same set up without the partitions, so we'll need a robust solution for avoiding switched tracks. (We'll also be able to take advantage of tattoos to propagate tracks by identification, but I'd like the initial tracklets to be as clean as possible.)

Do people have any other tips for improving tracking? I see that I could catch these in proofreading, but we'll be scaling this up to a level where correcting many tracks by hand probably isn't feasible, and certainly isn't desirable.

Thanks!

(see example below, from one frame to the next, track_0 jumps from top right to bottom left. In later frames, track_13 jumps to top right)

[talmo]

Hi @aperkes,

This is definitely a bit tricky in the existing framework. I agree that velocity is usually a clear signal for switches, but how do we incorporate it?

Large displacements should result in a large matching cost so velocity is technically considered, but whether it results in a match or not might depending on the matching method, window size, whether and how many animals appear/disappear, and if there are pose estimation errors.

Imposing a hard constraint would also be tricky since that involves resorting to integer linear program (ILP) solvers which can take multiple, possibly combinatorially intractable sets of constraints.

All that said, we should definitely have some way to use this form of filtering, even if it won't work in every case or requires more user defined parameters. I'll move this over to the ideas board and see how we can come up with a reasonable solution that works for your case.

[cocoa1231]

Hello. I'm facing the same issue right now. Has there been any updates regarding this?

[scowlingwind]

Hello @talmo, I'm tracking zebrafish larvae using SLEAP v1.3.0 and am facing the same issue with tracks being swapped between fish ( see screenshot below). See also the plot with the tracks of the head node across the video which is quite noisy. Any way to fix this rather than manually correcting track identities frame-by-frame?