Skip to content

Fix CV->GL pose conversion in updateTrackable()#36

Merged
kalwalt merged 1 commit into
webarkit:devfrom
kalwalt:fix/pose-cv-to-gl
May 30, 2026
Merged

Fix CV->GL pose conversion in updateTrackable()#36
kalwalt merged 1 commit into
webarkit:devfrom
kalwalt:fix/pose-cv-to-gl

Conversation

@kalwalt

@kalwalt kalwalt commented May 30, 2026

Copy link
Copy Markdown
Member

Summary

Stop applying a partial CV→GL conversion in WebARKitPatternTrackingInfo::updateTrackable(). Today the rotation columns Y and Z are negated but the translation isn't — an inconsistent half-conversion that leaves consumers with a pose where translation Z stays positive. Under a standard OpenGL projection (which expects objects in front to have negative Z in view space), tracked objects clip behind the camera and never render.

After this change, getPoseMatrix2() / trans carry the raw OpenCV pose (rotation unmodified, translation kept at the existing 0.001 * 1.64 scale). The CV→GL handedness flip (negate Y and Z rows including translation) is then applied uniformly downstream by arglCameraViewRHf, producing a mathematically clean diag(1, -1, -1, 1) * pose modelview that places the tracked object in front of the GL camera.

Both downstream paths benefit:

  • JS side: WebARKitController.process_rawarglCameraViewRHf(pose, ...) → correct matrixGL_RH.
  • C++ side: WebARKitManager::getGLViewMatrix()arglCameraViewRHf(getPoseMatrix2(), ...) → correct modelview.

Surfaced and verified downstream by webarkit/webarkit-testing#31 / #32 (static-image Teblid example): with this fix the AR overlay is no longer frustum-clipped behind the camera.

Breaking change note

This changes the runtime semantics of pose / trans for any consumer that relied on the previous partially-converted shape. Consumers rendering with OpenGL/three.js should now use matrixGL_RH (or getGLViewMatrix() in C++) as the GL-ready modelview, and treat pose as the raw OpenCV camera pose ([R | t], +Z forward).

Diff

A single 4-line hunk in WebARKit/WebARKitPattern.cpp (drop two - signs, plus a comment explaining the convention).

Test plan

  • Existing gtests still pass (no projection-matrix value asserted against this code path).
  • Downstream: rebuild WASM, load examples/threejs_teblid_static_image_ES6_example.html from Add static image example with Teblid tracker (#30) webarkit-testing#32 (after the companion PR), confirm sphere appears anchored to the marker rather than clipped behind the camera.

Related

🤖 Generated with Claude Code

Stop negating rotation columns 1 and 2 in updateTrackable(). The previous
behavior applied a partial CV->GL conversion (rotation columns flipped,
translation untouched), which left consumers with an inconsistent pose:
the rotation was neither raw OpenCV nor a clean GL right-handed modelview,
and the translation Z stayed positive — causing tracked objects to render
behind the camera in OpenGL/three.js scenes (clip.w < 0, ndc.z > 1).

After this change, getPoseMatrix2() / trans returns the raw OpenCV pose
(rotation unmodified, translation kept at the existing 0.001 * 1.64 scale).
The CV->GL handedness flip (negate Y and Z rows including translation) is
applied downstream by arglCameraViewRHf when building the GL right-handed
modelview (matrixGL_RH on the JS side, getGLViewMatrix() in WebARKitManager
on the C++ side). Both downstream paths are now mathematically clean
diag(1, -1, -1, 1) * pose conversions and produce a correct modelview that
places the tracked object in front of the GL camera.

Note: this is a semantic change to `pose` / `trans` for any consumer that
relied on the previous partially-converted shape. Downstream consumers
should use matrixGL_RH (or getGLViewMatrix) for OpenGL rendering and treat
`pose` as the raw OpenCV camera pose.

Refs: webarkit#34, webarkit/webarkit-testing#31

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
@kalwalt kalwalt self-assigned this May 30, 2026
@kalwalt kalwalt added bug Something isn't working enhancement New feature or request C/C++ code concerning the C/C++ code design and improvements Emscripten labels May 30, 2026
@kalwalt kalwalt moved this from In progress to Review in progress in New markerless image tracking May 30, 2026
@kalwalt kalwalt moved this from Review in progress to Reviewer approved in New markerless image tracking May 30, 2026
@kalwalt kalwalt merged commit 3429eb3 into webarkit:dev May 30, 2026
@github-project-automation github-project-automation Bot moved this from Reviewer approved to Done in New markerless image tracking May 30, 2026
kalwalt added a commit to webarkit/webarkit-testing that referenced this pull request May 30, 2026
webarkit/WebARKitLib#36 was squash-merged into upstream/dev as 3429eb3
(same content as the prior dab5073 SHA, different hash due to the squash).
Re-pin the submodule pointer to the canonical commit so a fresh `git
submodule update --init` against the .gitmodules URL (canonical
webarkit/WebARKitLib) resolves successfully.

Refs: #31, webarkit/WebARKitLib#34

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
@kalwalt kalwalt deleted the fix/pose-cv-to-gl branch May 30, 2026 11:49
kalwalt added a commit to webarkit/webarkit-testing that referenced this pull request May 30, 2026
webarkit/WebARKitLib#36 was squash-merged into upstream/dev as 3429eb3
(same content as the prior dab5073 SHA, different hash due to the squash).
Re-pin the submodule pointer to the canonical commit so a fresh `git
submodule update --init` against the .gitmodules URL (canonical
webarkit/WebARKitLib) resolves successfully.

Refs: #31, webarkit/WebARKitLib#34

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
kalwalt added a commit to webarkit/webarkit-testing that referenced this pull request Jun 21, 2026
webarkit/WebARKitLib#36 was squash-merged into upstream/dev as 3429eb3
(same content as the prior dab5073 SHA, different hash due to the squash).
Re-pin the submodule pointer to the canonical commit so a fresh `git
submodule update --init` against the .gitmodules URL (canonical
webarkit/WebARKitLib) resolves successfully.

Refs: #31, webarkit/WebARKitLib#34

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working C/C++ code concerning the C/C++ code design and improvements Emscripten enhancement New feature or request

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

1 participant