[tf-core] Cannot pass ImageBitmap to model in worker

[AlexShafir]

System information

• Have I written custom code (as opposed to using a stock example script provided in TensorFlow.js): yes
• OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Windows 7
• TensorFlow.js installed from (npm or script link): https://www.jsdelivr.com/package/npm/@tensorflow/tfjs-core
• TensorFlow.js version (use command below): 3.9.0
• Browser version: Chrome 94.0.4606.81
• Tensorflow.js Converter Version: 3.9.0

Model: face-landmarks-detection v. 0.0.4, with wasm-only backend.

Preconditions

1. Put all face-landmarks-detection code into web-worker
2. Pass current ImageBitmap

Describe the current behavior
ReferenceError: document is not defined.

Describe the expected behavior
ImageBitmap gets processed.

Standalone code to reproduce the issue
Provide a reproducible test case that is the bare minimum necessary to generate
the problem. If possible, please share a link to Colab/CodePen/any notebook.
https://github.com/AlexShafir/Sensoria/blob/tfjs5743
Message passed here, received here

Other info / logs Include any logs or source code that would be helpful to
diagnose the problem. If including tracebacks, please include the full
traceback. Large logs and files should be attached.

Uncaught (in promise) ReferenceError: document is not defined
    at fromPixels_ (:8887/fld/tf-core.js:8911)
    at Object.fromPixels__op [as fromPixels] (:8887/fld/tf-core.js:5406)
    at :8887/fld/face-landmarks-detection.js:1350
    at :8887/fld/tf-core.js:4394
    at Engine.scopedRun (:8887/fld/tf-core.js:4404)
    at Engine.tidy (:8887/fld/tf-core.js:4393)
    at Object.tidy (:8887/fld/tf-core.js:10072)
    at FaceMesh.<anonymous> (:8887/fld/face-landmarks-detection.js:1348)
    at step (:8887/fld/face-landmarks-detection.js:81)
    at Object.next (:8887/fld/face-landmarks-detection.js:62)

Probably related to #4218

[vladmandic]

This is not a bug in TFJS, this is how web workers work in general - there is no DOM in a worker, so you cannot rely on any DOM objects. TFJS could print more informative error message, but root cause is the same.

What you can do is pass pixel data (as value or actually transfer ownership of an array to worker) such as ImageData.data (which is just an array so that can be used by web worker) that you've already prepared in the main thread.

[AlexShafir]

@vladmandic

• Sure, web workers do not have access to document.
• Thanks for tip with ImageData.data! I ended up using ImageData.data.buffer as ArrayBuffer is transferable.

[vladmandic]

@AlexShafir yup, thats the best approach for web workers...
and inside web worker you can either reconstruct ImageData and then use fromPixels or just create tensor directly (but then you need to slice off alpha channel manually since ImageData is in RGBA format)

[rthadur]

Thank you @vladmandic @AlexShafir closing this issue as this is not a bug and more support issue. Feel free to @mention to reopen.

[google-ml-butler]

Are you satisfied with the resolution of your issue?
Yes
No

[AlexShafir]

@rthadur if API says that one can enter ImageBitmap, you pass it (in worker context) and it crashes, I expect this behavior is called a bug?
In any case, what is the process for this kind of problems?

[google-ml-butler]

Are you satisfied with the resolution of your issue?
Yes
No

[google-ml-butler]

Are you satisfied with the resolution of your issue?
Yes
No