[tfjs-models] - Inconsistent detections

[badman-rodriguez]

TensorFlow.js version

Versions:

    "@tensorflow-models/coco-ssd": "^2.0.2",
    "@tensorflow/tfjs-converter": "^1.7.2",
    "@tensorflow/tfjs-core": "^1.7.2",

Browser version

Chrome: 81.0.4044.92 (Official Build) (64-bit)

The issue is a little weird but it happens all the time.

How to replicate

Send b64 image for Object detection... and it has 0 detections

console.log output:

<img src=​"data:​image/​jpeg;​base64,............... b64 text" width=​"500" height=​"375">​
Dropzone.svelte:75 number of detections:  0
Dropzone.svelte:79 upload-and-predict: 145.814208984375ms

Same b64 image for object detection... and!

console.log output:

<img src=​"data:​image/​jpeg;​base64,............... b64 text" width=​"500" height=​"375">​
Dropzone.svelte:75 number of detections:  1
Dropzone.svelte:79 upload-and-predict: 127.974853515625ms

and it has 1 detection...
This happens 100% of the time.

Extra Note: It has to be the same image twice. If i keep adding different images it wont predict until i load the same one a second time.
Seems like it needs a second time around to get a detection in place.

Code

    import * as cocoSsd from '@tensorflow-models/coco-ssd';
    .
    .
    .
    const baseModel = 'mobilenet_v1';
    model = await cocoSsd.load({ base: baseModel });
    const detectionTest = new Image();
    detectionTest.src = b64Image; // This is b64 string like in the screen shot
    detectionTest.width = 500; // Have to manually set this... or it leads to a diff bug 
    detectionTest.height = 375; // Have to manually set this... or it leads to a diff bug
    bug is this: https://github.com/tensorflow/tfjs/issues/322
    const result = await model.detect(detectionTest);
    console.log('number of detections: ', result.length);
   .
   .
   .

[rthadur]

@badman-rodriguez can we have a codepen example or share the images to reproduce ?

[badman-rodriguez]

@rthadur thanks for responding so quickly. Will set up something pretty and share a link for you in a bit.

Thank you.

[badman-rodriguez]

@rthadur here is a sample of the behavior:
https://github.com/badman-rodriguez/svelte-tfjs-imageupload

Feel free to clone and give feedback as needed.

[rthadur]

@badman-rodriguez thanks for the info , it looks like you are using a java script framework(svelte),I tried original coco-ssd example it is working as expected , I believe this is not a bug or feature request related to tfjs which we can help here , I would suggest you ask this question in stack-overflow with tag 'tensorflow.js' , thank you

[badman-rodriguez]

that's easy to blame the framework..... once i solve the problem. will post the actual solution here. Figured an expert with this package might have some expect insight on some causes....

welp, appreciate the due diligence in the investigation.