Fix crash when a marker is detected on the first frame (#37)#38
Merged
Merged
Conversation
processFrame() could call GetTrackedFeaturesWarped() with an empty tracked- point selection on the very first frame: the optical-flow branch that calls GetInitialFeatures() (which populates _selectedPts) is gated on _frameCount > 0, yet MatchFeatures() can set _isDetected = true on frame 0. cv::perspectiveTransform then asserts on the empty input (scn+1 == m.cols, i.e. 2 == 3) and throws. Two guards: - TrackingPointSelector::GetTrackedFeaturesWarped() returns an empty result when the selection is empty or the homography is empty, instead of calling perspectiveTransform on empty input. - processFrame()'s pose block only runs solvePnP when there are >= 4 correspondences and the image/object point counts match. A live camera masks this (detection essentially never succeeds on the literal first frame); a static image triggers it deterministically. The webarkit-testing static example previously worked around it with a blank warmup frame; this fixes it at the source. Refs: webarkit#37, webarkit/webarkit-testing#30 Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
kalwalt
added
bug
github-project-automation
Bot
moved this to In progress
in New markerless image tracking
kalwalt
moved this from In progress
to Review in progress
in New markerless image tracking
github-project-automation
Bot
moved this from Review in progress
to Done
in New markerless image tracking
kalwalt
added a commit
to webarkit/webarkit-testing
that referenced
this pull request
May 30, 2026
Bumps the WebARKitLib submodule to the canonical dev SHA after webarkit/WebARKitLib#38 (squash) merged the first-frame perspectiveTransform crash fix, and commits the rebuilt dist/WebARKit.js and build/ WASM artifacts. The static Teblid example no longer relies on the warmup frame to avoid the frame-0 crash (the library now guards it); the warmup is retained only to smooth the static-image detection->tracking handoff. Refs: webarkit/WebARKitLib#37, #30 Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This was referenced Jun 15, 2026
kalwalt
added a commit
to webarkit/webarkit-testing
that referenced
this pull request
Jun 21, 2026
Bumps the WebARKitLib submodule to the canonical dev SHA after webarkit/WebARKitLib#38 (squash) merged the first-frame perspectiveTransform crash fix, and commits the rebuilt dist/WebARKit.js and build/ WASM artifacts. The static Teblid example no longer relies on the warmup frame to avoid the frame-0 crash (the library now guards it); the warmup is retained only to smooth the static-image detection->tracking handoff. Refs: webarkit/WebARKitLib#37, #30 Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes #37.
processFrame()could callGetTrackedFeaturesWarped()with an empty tracked-point selection on the very first frame, makingcv::perspectiveTransformassert and throw.Root cause: the optical-flow branch that calls
GetInitialFeatures()(which populates_selectedPts) is gated on_frameCount > 0, butMatchFeatures()can set_isDetected = trueon frame 0. The pose block (if (_isDetected || _isTracking)) then callsGetTrackedFeaturesWarped()on the empty selection →perspectiveTransformhitsCV_Assert(scn + 1 == m.cols)(empty input → 1-channel Mat →2 == 3).A live camera masks this (detection essentially never succeeds on the literal first frame); a static image triggers it deterministically.
Changes
TrackingPointSelector::GetTrackedFeaturesWarped()— return an empty result when the selection or the homography is empty, instead of callingperspectiveTransformon empty input (fixes the crash at the source).WebARKitTracker::processFrame()— only run pose estimation when there are>= 4correspondences and the image/object point counts match (solvePnPneeds ≥4; avoids estimating from an empty/degenerate set).Both are defensive and do not change behavior on frames where the selection is properly populated.
Verification
Built to WASM and run in the webarkit-testing static Teblid example with its warmup frame removed, so the real image is processed on frame 0 and exercises the exact crash path:
cv::error/perspectiveTransformexception (stack:GetTrackedFeaturesWarped→processFrame).Existing examples (with their normal frame flow) are unaffected.
Notes
The webarkit-testing static example currently feeds a blank warmup frame as a workaround; with this fix the library is robust against frame-0 detection for any consumer, so the warmup is no longer required for crash-avoidance (it's retained there only to smooth the static-image detection→tracking handoff).
Refs: webarkit/webarkit-testing#30, #31
🤖 Generated with Claude Code