Merged
Conversation
Work-in-progress changes to SAM2 support including Hiera backbone refinements, benchmark improvements, and plan updates. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
PABannier
force-pushed
the
wip/sam2-improvements
branch
from
b9399e9 to
52eab22
Compare
The cached graph and tensor_copy_async optimizations in the WIP commit caused tracking masks to degrade to zero within 5 frames. Reverting to the working (non-cached) implementation until these optimizations can be properly debugged. The three video tracking bug fixes from the previous commit remain intact. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Adds sam3_params.encode_img_size to override the model's native image resolution at runtime (e.g. 512 instead of 1024). When set, the Hiera backbone, FPN neck, SAM decoder, memory encoder, and all PE caches use the effective resolution, producing ~4x faster encoding at half the spatial dimensions. All pipeline functions (propagate_single, encode_memory, segment_pvs, build_prompt_and_pos, build_sam_dec_graph, populate_pe_cache, ensure_tracker_pe_caches) now accept or derive effective feat_size from state rather than hardcoding hp.feat_size(). Verified: tracking works at both 1024 (native) and 512 (half-res) on SAM2.1 tiny f32 with the bedroom video. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
main_video.cpp was not parsing --encode-img-size, so the parameter was always 0 (default = native resolution). Also fixed the SAM3 ViT encode path to use sam3_eff_img_size() instead of hardcoded hp.img_size. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Test plan
🤖 Generated with Claude Code