This is a project to count and detect Daphnia (water fleas) automatically from grayscale videos. It is particularly tuned towards videos I was using (which are not mine, so I can't make them available), and so it will not generalize to other videos without modification. It looks for roughly constant velocity dark blotches on a lighter stationary background. It can tolerate gradual overall changes in background lighting, but will have trouble with rapid flickering.
It works by implementing a multi-object tracking algorithm. In short, frame-by-frame detections, then association of those detections to existing objects and track merging/creation/deletion, and then Kalman filtering with linear dynamics).
I've made efforts to make the code modular and readable, so hopefully someone finds it useful.
How to use
Please note that Python 3 is required. To use this to count objects in a video:
git clone firstname.lastname@example.org:nuchi/multi_object_tracking.git cd multi_object_tracking python -m pip install -e . # install prerequisites, then install the project in editable mode mkdir output # create a directory for debug output python -m multi_object_tracking \ --path $PATH_TO_VIDEOS_DIRECTORY \ --out output.txt \ --debug-out output
See also the other command-line options with
python -m multi_object_tracking --help
--debug-output $OUTPUT_DIRECTORY option writes json-formatted debug output to a directory, which can be used to construct visualizations. See the notebook
debug.ipynb which can be used to display the visualizations.
To evaluate the performance against a labeled video, run
python -m multi_object_tracking.evaluate --input list_of_videos.txt --use-filters
list_of_videos.txt is a file that looks like:
<path_to_video_1>,<path_to_directory_with_labels_for_video_1> <path_to_video_2>,<path_to_directory_with_labels_for_video_2> ...
The labels for a video
my-video.avi should have names like
my-video_0123_1_labelbasic.json. I haven't documented the label format, because I was working with existing files that were handed to me. If you're finding this project, it might be best to rewrite the evaluation section for your own purposes rather than try to adapt mine.
How it works
In broad strokes:
- Detect objects in a single frame ("detections"). (
- Associate to each existing track (a constant velocity Kalman filter) a best detection (
- Update each track which received a detection (
- Create new tracks for detections that didn't correspond to an existing track (
- Delete tracks whenever they too closely mirror a "better" track (i.e. one with more certainty) (
- Delete tracks that haven't been observed recently (
The whole business is orchestrated in
track.py, which is called via
debug.ipynb for how to use the
visualize.py module. Model parameters are stored in
default_params.json and can be overridden with another file via
python -m multi_object_tracking[.evaluate] --params <path_to_alternate_params> or by calling
- Improve the association phase by using
scipy.optimize.linear_sum_assignmentand minimizing total negative-log-likelihood.