tamserve 0.2.0

tamserve now runs on as many GPUs as you have, on one machine or many, reads your cameras for you, and keeps going when cameras or machines fail.

What's new

• Frames straight from the decoder. engine.step() takes each frame exactly as the decoder gives it, and the engine uploads and resizes it on the GPU. The README's live loop went from 51 to 273 frames a second with eight 720p cameras on an RTX 4090, against 330 for the engine on its own.
• Several GPUs with a Router. It runs a worker on each GPU, puts each new camera on the least busy one, and gives you every camera's results in one stream. Two RTX 4090s tracked 476 frames a second from 32 cameras, against 241 on one.
• GPUs on other machines. Start tamserve worker on each GPU machine and give the router their addresses. Workers prove they know a shared token without sending it, and messages are plain data, so neither side ever runs anything it is sent. Two machines, in California and Utah, tracked 494 frames a second together, 98% of what they did separately.
• It keeps going, and says so. A dropped camera is reopened with growing waits and keeps its objects. A worker that stops answering is noticed within 15 seconds, its cameras move to the other workers, and it is brought back when it can be. Every change arrives as a status update in results(), goes to the tamserve logger, and shows in router.status().

Fixes

• Compiled engines could fail to start. Python's garbage collector, running while compiled code was being recorded as a CUDA graph, could make CUDA abandon the recording. It happened in some processes every time and in others never. The engine now pauses the collector during each step. This affects 0.1.0 with compile=True.
• SAM 2.1 weights download again. Meta's download server let its certificate lapse on 12 November. tamserve now falls back to Meta's own copies on Hugging Face, which are checked against the same pinned SHA-256 digests.

Changes to check before upgrading

• prepare_batch() and prepare(..., resize_on_device=True) are gone: pass step() the frame as decoded. prepare() remains for masks that must match the original code bit for bit.

Install

pip install tamserve
pip install opencv-python-headless   # to open cameras by address with a Router

To try it in two minutes on a free GPU, open the Colab notebook. Every number above, and how it was measured, is in the README and docs/results.md.

tamserve 0.1.0

The first release of tamserve, a serving engine for SAM 2-family video trackers. One copy of the model tracks every object on every camera, live, on a single GPU.

What is in it

• Engine, with one call per round: step() takes the newest frame from each camera and returns a mask for every tracked object. You can start tracking something, stop, or point at an object again on any frame, and cameras can come and go.
• EdgeTAM and the four SAM 2.1 models. They are downloaded the first time you use them, at a pinned version, and verified before anything is loaded.
• compile=True, which compiles the model for your GPU and is what the speed figures use.
• The tests and benchmarks behind every number in the README.

Measured on one RTX 4090

	the model's own code	tamserve
EdgeTAM, 1 camera	177 frames/s	219 frames/s
EdgeTAM, 8 cameras	280 frames/s	323 frames/s
EdgeTAM, GPU memory at 8 cameras	8.6 GB	2.2 GB
SAM 2.1 tiny, 8 cameras	89 frames/s	91 frames/s
SAM 2.1 tiny, GPU memory at 8 cameras	10.0 GB	3.4 GB

Accuracy on DAVIS 2017 is unchanged for all five models. The model's own code is EdgeTAM's repository for EdgeTAM and Meta's current sam2 package for SAM 2.1, both compiled and run as one process per camera. The README has the full tables and how they were measured.

Install

pip install git+https://github.com/aryamandarda/tamserve.git@v0.1.0

You need Python 3.10 or newer and an NVIDIA GPU. tamserve is not on PyPI yet.