New ARToolKitNFT class in the C++ Emscripten code #301

kalwalt · 2023-03-29T12:09:30Z

I will rewrite the C++ code, following this structure:

Create an ARToolKitNFT class and export it thanks to Emscripten bindings to WASM, instead of the function export in an extern C as was before.

Consequently change the Typescript, code i will add these functions/changes:

- see issue #292

src/index_improved.ts

- removing decorate() for instantitating methods - static properties exposed explictly - all methods declared inside ARToolkitNFT class

- removing some not necessary properties in ARControllerNFT_improved

- webpack and typescript

kalwalt · 2023-04-01T17:52:16Z

I think i have implemented all the planned features, i'll make a last check to be sure that it is ready to be merged.

kalwalt · 2023-04-02T17:00:34Z

Sometimes testing the new lib, i get this error:

Uncaught RuntimeError: null function or function signature mismatch
    at 001c84ce:0x5966
    at 001c84ce:0x3ca2a
    at 001c84ce:0x5fb60
    at 001c84ce:0xd44d
    at fA.ARToolKitNFT$detectNFTMarker [as detectNFTMarker] (eval at _A (ARToolkitNFT_simd.js:2:678066), <anonymous>:8:10)
    at A.value (ARToolkitNFT_simd.js:2:739639)
    at A.value (ARToolkitNFT_simd.js:2:746781)
    at A.value (ARToolkitNFT_simd.js:2:746000)
    at process (artoolkitNFT_ES6_gray.worker.js:151:8)
    at self.onmessage (artoolkitNFT_ES6_gray.worker.js:47:7)

When i recevive this error, after it's impossible to continue to detect and track. It looks strange, at first sight i was thinking that the reason was some bad bug in the typescript code ( a void function instead of function returning a number) but seems a silent issue instead in WASM (??) I think i nedd to debug adding some flags in makem.js (-g4 and other debug symbols).

kalwalt · 2023-04-02T18:48:28Z

Added some flags to have more info about this issue, running the ARToolkitNFT_ES6_example.html i got now:

artoolkitNFT_ES6.worker.js:125 RuntimeError: memory access out of bounds
    at icpDeleteHandle (003d3f82:0x11827)
    at ARToolKitNFT::deleteHandle() (003d3f82:0x29669)
    at ARToolKitNFT::setCamera(int, int) (003d3f82:0x29dc4)
    at ARToolKitNFT::setup(int, int, int) (003d3f82:0x32936)
    at emscripten::internal::MethodInvoker<int (ARToolKitNFT::*)(int, int, int), int, ARToolKitNFT*, int, int, int>::invoke(int (ARToolKitNFT::* const&)(int, int, int), ARToolKitNFT*, int, int, int) (003d3f82:0x39c44)
    at PA.ARToolKitNFT$setup [as setup] (eval at zI (ARToolkitNFT_simd.js:2:1410886), <anonymous>:11:10)
    at A.value (ARToolkitNFT_simd.js:2:1486634)
    at A.<anonymous> (ARToolkitNFT_simd.js:2:1498899)
    at s (ARToolkitNFT_simd.js:2:1015)
    at Generator.<anonymous> (ARToolkitNFT_simd.js:2:2370)
onError @ artoolkitNFT_ES6.worker.js:125
Promise.catch (asinc)
load @ artoolkitNFT_ES6.worker.js:133
self.onmessage @ artoolkitNFT_ES6.worker.js:39

Note deleteHandle is:

jsartoolkitNFT/emscripten/ARToolKitNFT_js.cpp

Lines 184 to 198 in d3e95ad

    
           void ARToolKitNFT::deleteHandle() { 
        
             if (this->arhandle != NULL) { 
        
               arPattDetach(this->arhandle); 
        
               arDeleteHandle(this->arhandle); 
        
               this->arhandle = NULL; 
        
             } 
        
             if (this->ar3DHandle != NULL) { 
        
               ar3DDeleteHandle(&(this->ar3DHandle)); 
        
               this->ar3DHandle = NULL; 
        
             } 
        
             if (this->paramLT != NULL) { 
        
               arParamLTFree(&(this->paramLT)); 
        
               this->paramLT = NULL; 
        
             } 
        
           }

icpDeleteHandle is invoked by

jsartoolkitNFT/emscripten/ARToolKitNFT_js.cpp

Line 191 in d3e95ad

ar3DDeleteHandle(&(this->ar3DHandle));

in our implementation

jsartoolkitNFT/emscripten/ARToolKitNFT_js.h

Line 93 in d3e95ad

AR3DHandle *ar3DHandle;

we not assign a value during the initialization in the constructor. I tested and if init AR3DHandle *ar3DHandle to NULL this issue is avoided.
In general i think it's better to init the parameter in the constructor, i think i will opt to this code design.

kalwalt · 2023-04-02T19:39:14Z

I solved the issue above initializing the private fields in the .cpp file, now i got another error (while testing ARToolkitNFT_ES6_example.html) related to the passVideoData function. Exactly i get an heap-buffer-overflow on address 0x16204d00 in more specific it points to the memcpy invocations inside the passVideoData

jsartoolkitNFT/emscripten/ARToolKitNFT_js.cpp

Lines 38 to 49 in 176cece

    
           int ARToolKitNFT::passVideoData(emscripten::val videoFrame, 
        
                                           emscripten::val videoLuma) { 
        
             std::vector<uint8_t> vf = 
        
                 emscripten::convertJSArrayToNumberVector<uint8_t>(videoFrame); 
        
             std::vector<uint8_t> vl = 
        
                 emscripten::convertJSArrayToNumberVector<uint8_t>(videoLuma); 
        
             memcpy(this->videoFrame, vf.data(), sizeof(uint8_t) * this->videoFrameSize); 
        
             memcpy(this->videoLuma, vl.data(), 
        
                    sizeof(uint8_t) * this->videoFrameSize / 4); 
        
             return 0; 
        
           }

I receive an analog error with the basic.html, but without the additional infos because i added the -fsanitize=address and debug symbols only to the ES6 lib. But i'm quite sure that is caused by this.
Anyway it is not caused by this PR but this is a silent issue to be checked.

gist

kalwalt · 2023-04-02T20:30:21Z

this should be better:

// ARToolKitNFT_js.cpp
// inside passVideoData()
// instead of memcpy() stuff...
this->videoFrame = vf.data();
this->videoLuma = vl.data();

// ARToolKitJS.cpp
// inside passVideoData()
// instead of memcpy() stuff...
arc->videoFrame = vf.data();
arc->videoLuma = vl.data();

This works but i need to add -sSTACK_SIZE=20mb otherwise i got a stack overflow exception, maybe because the other flags?
I will test removing all the debug flags.

Why i used memcpy? i can not remember the reason...

- avoid memcpy in passVideoData

kalwalt · 2023-04-02T21:12:27Z

Without the debug flags i don't need to add the -sSTACK_SIZE=20mb, i don't encounter any other issue for now. but i need to report it in the issue tracker to take trace of it. I did most of the time test on a desktop machine i will test on Android to see if it is ok.

kalwalt · 2023-04-03T21:23:34Z

Debugging ES6 examples in Android i get this error:

artoolkitNFT_ES6.worker.js:125 RuntimeError: Aborted(RangeError: WebAssembly.instantiate(): Out of memory: Cannot allocate Wasm memory for new instance). Build with -sASSERTIONS for more info.

I got this in the past but i need to understand if it is caused by new changes.

kalwalt · 2023-04-03T21:34:15Z

adding an -sASSERTIONS=2 flag to only he ES6 version of the lib i get a stack overflow:

artoolkitNFT_ES6_wasm.js:8 Aborted(stack overflow (Attempt to set SP to 0x0000de00, with stack limits [0x00011770 - 0x00021770]). If you require more stack space build with -sSTACK_SIZE=<bytes>)
abort @ artoolkitNFT_ES6_wasm.js:8
___handle_stack_overflow @ artoolkitNFT_ES6_wasm.js:8
$func60 @ 001e4732:0x1c7f
$func309 @ 001e4732:0x2bdc9
$func366 @ 001e4732:0x2f5f9
ARToolKitNFT$detectMarker @ VM13:8
detectMarker @ ARToolkitNFT.ts:237
detectMarker @ ARControllerNFT.ts:400
process @ ARControllerNFT.ts:280
process @ artoolkitNFT_ES6.worker.js:140
self.onmessage @ artoolkitNFT_ES6.worker.js:44
artoolkitNFT_ES6_wasm.js:8 Aborted(stack overflow (Attempt to set SP to 0x0000dd10, with stack limits [0x00011770 - 0x00021770]). If you require more stack space build with -sSTACK_SIZE=<bytes>)
abort @ artoolkitNFT_ES6_wasm.js:8
___handle_stack_overflow @ artoolkitNFT_ES6_wasm.js:8
$func60 @ 001e4732:0x1c7f
$func309 @ 001e4732:0x2bdc9
$func366 @ 001e4732:0x2f5f9
ARToolKitNFT$detectMarker @ VM13:8
detectMarker @ ARToolkitNFT.ts:237
detectMarker @ ARControllerNFT.ts:400
process @ ARControllerNFT.ts:280
process @ artoolkitNFT_ES6.worker.js:140
self.onmessage @ artoolkitNFT_ES6.worker.js:44
artoolkitNFT_ES6_wasm.js:8 Aborted(stack overflow (Attempt to set SP to 0x0000dc20, with stack limits [0x00011770 - 0x00021770]). If you require more stack space build with -sSTACK_SIZE=<bytes>)
abort @ artoolkitNFT_ES6_wasm.js:8
___handle_stack_overflow @ artoolkitNFT_ES6_wasm.js:8
$func60 @ 001e4732:0x1c7f
$func309 @ 001e4732:0x2bdc9
$func366 @ 001e4732:0x2f5f9
ARToolKitNFT$detectMarker @ VM13:8
detectMarker @ ARToolkitNFT.ts:237
detectMarker @ ARControllerNFT.ts:400
process @ ARControllerNFT.ts:280
process @ artoolkitNFT_ES6.worker.js:140
self.onmessage @ artoolkitNFT_ES6.worker.js:44
artoolkitNFT_ES6_wasm.js:8 Aborted(stack overflow (Attempt to set SP to 0x0000db30, with stack limits [0x00011770 - 0x00021770]). If you require more stack space build with -sSTACK_SIZE=<bytes>)
abort @ artoolkitNFT_ES6_wasm.js:8
___handle_stack_overflow @ artoolkitNFT_ES6_wasm.js:8
$func60 @ 001e4732:0x1c7f
$func309 @ 001e4732:0x2bdc9
$func366 @ 001e4732:0x2f5f9
ARToolKitNFT$detectMarker @ VM13:8
detectMarker @ ARToolkitNFT.ts:237
detectMarker @ ARControllerNFT.ts:400
process @ ARControllerNFT.ts:280
process @ artoolkitNFT_ES6.worker.js:140
self.onmessage @ artoolkitNFT_ES6.worker.js:44
artoolkitNFT_ES6_wasm.js:8 Uncaught RuntimeError: Aborted(stack overflow (Attempt to set SP to 0x0000de00, with stack limits [0x00011770 - 0x00021770]). If you require more stack space build with -sSTACK_SIZE=<bytes>)
    at abort (artoolkitNFT_ES6_wasm.js:8:14583)
    at ___handle_stack_overflow (artoolkitNFT_ES6_wasm.js:8:687719)
    at 001e4732:0x1c7f
    at 001e4732:0x2bdc9
    at 001e4732:0x2f5f9
    at ClassHandle.ARToolKitNFT$detectMarker [as detectMarker] (eval at new_ (artoolkitNFT_ES6_wasm.js:8:765162), <anonymous>:8:10)
    at ARToolkitNFT.detectMarker (ARToolkitNFT.ts:237:26)
    at ARControllerNFT.detectMarker (ARControllerNFT.ts:400:32)
    at ARControllerNFT.process (ARControllerNFT.ts:280:23)
    at process (artoolkitNFT_ES6.worker.js:140:8)
abort @ artoolkitNFT_ES6_wasm.js:8
___handle_stack_overflow @ artoolkitNFT_ES6_wasm.js:8
$func60 @ 001e4732:0x1c7f
$func309 @ 001e4732:0x2bdc9
$func366 @ 001e4732:0x2f5f9
ARToolKitNFT$detectMarker @ VM13:8
detectMarker @ ARToolkitNFT.ts:237
detectMarker @ ARControllerNFT.ts:400
process @ ARControllerNFT.ts:280
process @ artoolkitNFT_ES6.worker.js:140
self.onmessage @ artoolkitNFT_ES6.worker.js:44
artoolkitNFT_ES6_wasm.js:8 Uncaught RuntimeError: Aborted(stack overflow (Attempt to set SP to 0x0000dd10, with stack limits [0x00011770 - 0x00021770]). If you require more stack space build with -sSTACK_SIZE=<bytes>)
    at abort (artoolkitNFT_ES6_wasm.js:8:14583)
    at ___handle_stack_overflow (artoolkitNFT_ES6_wasm.js:8:687719)
    at 001e4732:0x1c7f
    at 001e4732:0x2bdc9
    at 001e4732:0x2f5f9
    at ClassHandle.ARToolKitNFT$detectMarker [as detectMarker] (eval at new_ (artoolkitNFT_ES6_wasm.js:8:765162), <anonymous>:8:10)
    at ARToolkitNFT.detectMarker (ARToolkitNFT.ts:237:26)
    at ARControllerNFT.detectMarker (ARControllerNFT.ts:400:32)
    at ARControllerNFT.process (ARControllerNFT.ts:280:23)
    at process (artoolkitNFT_ES6.worker.js:140:8)
abort @ artoolkitNFT_ES6_wasm.js:8
___handle_stack_overflow @ artoolkitNFT_ES6_wasm.js:8
$func60 @ 001e4732:0x1c7f
$func309 @ 001e4732:0x2bdc9
$func366 @ 001e4732:0x2f5f9
ARToolKitNFT$detectMarker @ VM13:8
detectMarker @ ARToolkitNFT.ts:237
detectMarker @ ARControllerNFT.ts:400
process @ ARControllerNFT.ts:280
process @ artoolkitNFT_ES6.worker.js:140
self.onmessage @ artoolkitNFT_ES6.worker.js:44
artoolkitNFT_ES6_wasm.js:8 Uncaught RuntimeError: Aborted(stack overflow (Attempt to set SP to 0x0000dc20, with stack limits [0x00011770 - 0x00021770]). If you require more stack space build with -sSTACK_SIZE=<bytes>)
    at abort (artoolkitNFT_ES6_wasm.js:8:14583)
    at ___handle_stack_overflow (artoolkitNFT_ES6_wasm.js:8:687719)
    at 001e4732:0x1c7f
    at 001e4732:0x2bdc9
    at 001e4732:0x2f5f9
    at ClassHandle.ARToolKitNFT$detectMarker [as detectMarker] (eval at new_ (artoolkitNFT_ES6_wasm.js:8:765162), <anonymous>:8:10)
    at ARToolkitNFT.detectMarker (ARToolkitNFT.ts:237:26)
    at ARControllerNFT.detectMarker (ARControllerNFT.ts:400:32)
    at ARControllerNFT.process (ARControllerNFT.ts:280:23)
    at process (artoolkitNFT_ES6.worker.js:140:8)
abort @ artoolkitNFT_ES6_wasm.js:8
___handle_stack_overflow @ artoolkitNFT_ES6_wasm.js:8
$func60 @ 001e4732:0x1c7f
$func309 @ 001e4732:0x2bdc9
$func366 @ 001e4732:0x2f5f9
ARToolKitNFT$detectMarker @ VM13:8
detectMarker @ ARToolkitNFT.ts:237
detectMarker @ ARControllerNFT.ts:400
process @ ARControllerNFT.ts:280
process @ artoolkitNFT_ES6.worker.js:140
self.onmessage @ artoolkitNFT_ES6.worker.js:44
artoolkitNFT_ES6_wasm.js:8 Uncaught RuntimeError: Aborted(stack overflow (Attempt to set SP to 0x0000db30, with stack limits [0x00011770 - 0x00021770]). If you require more stack space build with -sSTACK_SIZE=<bytes>)
    at abort (artoolkitNFT_ES6_wasm.js:8:14583)
    at ___handle_stack_overflow (artoolkitNFT_ES6_wasm.js:8:687719)
    at 001e4732:0x1c7f
    at 001e4732:0x2bdc9
    at 001e4732:0x2f5f9
    at ClassHandle.ARToolKitNFT$detectMarker [as detectMarker] (eval at new_ (artoolkitNFT_ES6_wasm.js:8:765162), <anonymous>:8:10)
    at ARToolkitNFT.detectMarker (ARToolkitNFT.ts:237:26)
    at ARControllerNFT.detectMarker (ARControllerNFT.ts:400:32)
    at ARControllerNFT.process (ARControllerNFT.ts:280:23)
    at process (artoolkitNFT_ES6.worker.js:140:8)
abort @ artoolkitNFT_ES6_wasm.js:8
___handle_stack_overflow @ artoolkitNFT_ES6_wasm.js:8
$func60 @ 001e4732:0x1c7f
$func309 @ 001e4732:0x2bdc9
$func366 @ 001e4732:0x2f5f9
ARToolKitNFT$detectMarker @ VM13:8
detectMarker @ ARToolkitNFT.ts:237
detectMarker @ ARControllerNFT.ts:400
process @ ARControllerNFT.ts:280
process @ artoolkitNFT_ES6.worker.js:140
self.onmessage @ artoolkitNFT_ES6.worker.js:44

kalwalt · 2023-04-03T21:49:59Z

if i enlarge the stack -sSTACK_SIZE=2mb it works for the first time but after the out of Memory error happens again and are affected all the examples. So maybe an issue only with my device? Simd or not Simd is the same. I may test with another device to see if there are differences. Note that on desktop i never had it.

- fix for issue #97 and webarkit/ARnft#212

kalwalt · 2023-04-04T15:17:33Z

last commit should fix issue RuntimeError: Aborted(RangeError: WebAssembly.instantiate(): Out of memory: Cannot allocate Wasm memory for new instance). and fix issue #97 and consequently will fix also webarkit/ARnft#212 but i will need to updated ARnft dependencies. This PR is ready to be merged.

kalwalt and others added 3 commits February 26, 2023 17:45

testing new c++ wrapper implementation

203293b

Merge branch 'dev' into feat-ARToolKitNFT-em-class

1ca0928

more methods to the ARToolKitNFT class

3d99aa5

kalwalt added enhancement New feature or request C/C++ emscripten labels Mar 29, 2023

kalwalt self-assigned this Mar 29, 2023

kalwalt added 3 commits March 29, 2023 15:09

last methods and constants to the improved_ES6

09f2c9b

Merge branch 'dev' into feat-ARToolKitNFT-em-class

deefff6

testing improved version

c4442b6

kalwalt mentioned this pull request Mar 29, 2023

Module '"@webarkit/jsartoolkit-nft"' has no exported member 'ARToolkitNFT' #292

Open

testing removing default export

d09e46a

- see issue #292

kalwalt commented Mar 29, 2023

View reviewed changes

src/index_improved.ts Outdated Show resolved Hide resolved

kalwalt and others added 8 commits March 29, 2023 23:19

testing if is better to leave two classes or not

ff09e0d

continue on two classes design

b3824c5

- removing decorate() for instantitating methods - static properties exposed explictly - all methods declared inside ARToolkitNFT class

convert properties to methods in IARToolkitNFT

60b7e2c

- removing some not necessary properties in ARControllerNFT_improved

restoring original names of the classes

59a8879

removing _improved version of the libs

3c23e48

new nftMarkers property

91b895d

upgrading packages

10f06cc

- webpack and typescript

avoid warning about willReadFrequently

1ccdf52

added license to new files

3d9526a

libs with some -g flags

d3e95ad

initializing vars in the cpp file

176cece

removing Debug flags

d8c3fc1

- avoid memcpy in passVideoData

adding missed destructor

5b252dc

lowering memory size

2debdda

- fix for issue #97 and webarkit/ARnft#212

kalwalt added dependencies Pull requests that update a dependency file Typescript labels Apr 4, 2023

update dependencies

ddb8b31

kalwalt merged commit a7bfcf3 into dev Apr 4, 2023
3 checks passed

kalwalt deleted the feat-ARToolKitNFT-em-class branch January 5, 2024 22:27

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

New ARToolKitNFT class in the C++ Emscripten code #301

New ARToolKitNFT class in the C++ Emscripten code #301

kalwalt commented Mar 29, 2023 •

edited

kalwalt commented Apr 1, 2023

kalwalt commented Apr 2, 2023

kalwalt commented Apr 2, 2023

kalwalt commented Apr 2, 2023

kalwalt commented Apr 2, 2023 •

edited

kalwalt commented Apr 2, 2023 •

edited

kalwalt commented Apr 3, 2023 •

edited

kalwalt commented Apr 3, 2023

kalwalt commented Apr 3, 2023 •

edited

kalwalt commented Apr 4, 2023

New ARToolKitNFT class in the C++ Emscripten code #301

New ARToolKitNFT class in the C++ Emscripten code #301

Conversation

kalwalt commented Mar 29, 2023 • edited

kalwalt commented Apr 1, 2023

kalwalt commented Apr 2, 2023

kalwalt commented Apr 2, 2023

kalwalt commented Apr 2, 2023

kalwalt commented Apr 2, 2023 • edited

kalwalt commented Apr 2, 2023 • edited

kalwalt commented Apr 3, 2023 • edited

kalwalt commented Apr 3, 2023

kalwalt commented Apr 3, 2023 • edited

kalwalt commented Apr 4, 2023

kalwalt commented Mar 29, 2023 •

edited

kalwalt commented Apr 2, 2023 •

edited

kalwalt commented Apr 2, 2023 •

edited

kalwalt commented Apr 3, 2023 •

edited

kalwalt commented Apr 3, 2023 •

edited