Question
I've found that transformers.js supports SAM as written in 2.14.0 release notes.
https://github.com/huggingface/transformers.js/releases/tag/2.14.0
I'm running the code on a M1 mac in a Brave browser.
But after I've used and adapted the example script, I can actually see in my browser console that the model is loaded and the browser is working.
But then suddenly it crashes with following error:
transformers.js:11821 Uncaught Error: An error occurred during model execution: "Missing the following inputs: input_points, input_labels.
My adapted code looks like this:
// using version 3.5.1
import {AutoProcessor, RawImage, SamModel} from "./node_modules/@huggingface/transformers/dist/transformers.js";
const model = await SamModel.from_pretrained('Xenova/slimsam-77-uniform');
const processor = await AutoProcessor.from_pretrained('Xenova/slimsam-77-uniform');
const img_url = 'https://huggingface.co/datasets/Xenova/transformers.js-docs/resolve/main/corgi.jpg';
const raw_image = await RawImage.read(img_url);
const input_points = [[[340, 250]]] // 2D localization of a window
const inputs = await processor(raw_image, input_points);
const outputs = await model(inputs); /// Error happens here
const masks = await processor.post_process_masks(outputs.pred_masks, inputs.original_sizes, inputs.reshaped_input_sizes);
console.log(masks);
// [
// Tensor {
// dims: [ 1, 3, 410, 614 ],
// type: 'bool',
// data: Uint8Array(755220) [ ... ],
// size: 755220
// }
// ]
const scores = outputs.iou_scores;
console.log(scores);
// Tensor {
// dims: [ 1, 1, 3 ],
// type: 'float32',
// data: Float32Array(3) [
// 0.8350210189819336,
// 0.9786665439605713,
// 0.8379436731338501
// ],
// size: 3
// }Markup:
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
</head>
<body>
<h1>SAM DEMO</h1>
<script src="main.js" type="module">
</script>
<pre id="pre"></pre>
</body>
</html>Can you maybe give me a hint what's the issue here or what I must e.g. change according to major version changes.
Thanks so much :-)
Question
I've found that transformers.js supports SAM as written in 2.14.0 release notes.
https://github.com/huggingface/transformers.js/releases/tag/2.14.0
I'm running the code on a M1 mac in a Brave browser.
But after I've used and adapted the example script, I can actually see in my browser console that the model is loaded and the browser is working.
But then suddenly it crashes with following error:
My adapted code looks like this:
Markup:
Can you maybe give me a hint what's the issue here or what I must e.g. change according to major version changes.
Thanks so much :-)