Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow tensor as input image #1396

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

lindapaiste
Copy link
Contributor

@lindapaiste lindapaiste commented May 22, 2022

We want any model which accepts an image input to also accept a 3D tensor of image pixels. This means that we should not call tf.browser.fromPixels directly with the input as it will be an error if called with a tensor. I am instead calling a new utility toTensor which is a simple wrapper around tf.browser.fromPixels that will first check if the input is already a tensor and just return it.

Change tf.browser.fromPixels to toTensor in:

  • BodyPix
  • CartoonGAN
  • Pix2Pix
  • StyleTransfer
  • UNET

There are chunks of logic in Darknet and Doodlenet for checking if the image is a tensor and conditionally converting it. These can be deleted entirely because the ImageClassifier handles converting the input to a tensor before calling the specific model's classify() function.


Two other minor changes:

  • I realized that await this.ready and await mediaReady should be awaited in parallel rather than sequentially so I changed that in a few places.
  • The existing imgToTensor function, which rescales and crops in addition to converting, should use 127.5 as the divisor rather than 127.

@lindapaiste
Copy link
Contributor Author

  • The existing imgToTensor function, which rescales and crops in addition to converting, should use 127.5 as the divisor rather than 127.

In hindsight I think that I was wrong about this. I got confused thinking that it should be half of 255, but I was forgetting that 0 is part of the range. 0 to 127 is 128 values and 0 to 255 is 256 values so it is already half.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant