wasm? #1

arogozhnikov · 2023-01-12T05:20:28Z

Hi Henry,
found this well-written implementation, great work.

Was looking for browser implementation of datagram scanner I could use in a lab.
So as an idea, maybe you'll be interested in compiling your library to web assembly. At least in theory, that should keep close-to-rust speed while allowing to use right from the browser.

Just an idea, feel free to close the issue.

hschimke · 2023-01-12T16:50:37Z

I'll look into this. I'm not particularly familiar with wasm at this point, so a little research would be necessary. Off the top of my head I see a few potential issues:

My limited understanding of wasm suggests that it works best with no_std libraries, which rxing currently is not. I have some future plans to move rxing to no_std, but those don't have a timeline yet.
The current version is really only well tested when using images from the image crate as an input source. I don't think that would be the preferred way to handle things in a wasm library, at least in part because the image crate is pretty large.

I'll look into it.

hschimke · 2023-01-13T18:28:57Z

I've done a little experimentation, though a bit more is needed. The library compiles to wasm just fine, and I was able to get a simple barcode generator working without too much work.

I'll play around with it and see if I can make a realistic wasm binding library for decode.

hschimke · 2023-01-13T18:55:09Z

Do you have a sample barcode you would want to decode? What sorts of formats are you looking at decoding, what input formats do you have available.

My preliminary idea is to make a very simple npm package that provides methods for:

Encoding barcodes to any of the supported formats, though what output format would be provided I haven't settled on.
Decoding a single barcode in an image defined by an array of luma8 values. This skips the (much less performant) code that handles multiple barcodes in a single image.

arogozhnikov · 2023-01-14T00:25:16Z

Single barcode is completely fine.

I am looking for ECC implementations that can handle DMRE: https://www.e-d-c.info/en/projects/dmre-en.html
It is the only format I found which is non-square, and for vials/tubes rectangular barcode is better. (Update: technically, PDF417 is too, but visually ECC DMRE is more compact)

This single-page example exposes image as a bytestream + dimensions, and I believe that's just RGB values interleaved. Converting to luminosity-only sounds reasonable

<!DOCTYPE html>
<!-- saved from url=(0059)https://usefulangle.com/demos/352/camera-capture-photo.html -->
<html><head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
<title>Demo - Capture Photo From Webcam Using Javascript</title>

<meta name="viewport" content="width=device-width,initial-scale=1.0,maximum-scale=1.0,user-scalable=no">
<style type="text/css">

button {
    width: 120px;
    padding: 10px;
    display: block;
    margin: 20px auto;
    border: 2px solid #111111;
    cursor: pointer;
    background-color: white;
}

#start-camera {
    margin-top: 50px;
}

#video {
    display: none;
    margin: 50px auto 0 auto;
}

#click-photo {
    display: none;
}

#dataurl-container {
    display: none;
}

#canvas {
    display: block;
    margin: 0 auto 20px auto;
}

#dataurl-header {
    text-align: center;
    font-size: 15px;
}

#dataurl {
    display: block;
    height: 100px;
    width: 320px;
    margin: 10px auto;
    resize: none;
    outline: none;
    border: 1px solid #111111;
    padding: 5px;
    font-size: 13px;
    box-sizing: border-box;
}

</style>
</head>

<body>

<button id="start-camera">Start Camera</button>
<video id="video" width="320" height="240" autoplay=""></video>
<button id="click-photo">Click Photo</button>
<div id="dataurl-container">
    <canvas id="canvas" width="320" height="240"></canvas>
    <div id="dataurl-header">Image Data URL</div>
    <textarea id="dataurl" readonly=""></textarea>
</div>

<script>

let camera_button = document.querySelector("#start-camera");
let video = document.querySelector("#video");
let click_button = document.querySelector("#click-photo");
let canvas = document.querySelector("#canvas");
let dataurl = document.querySelector("#dataurl");
let dataurl_container = document.querySelector("#dataurl-container");


function recognizeBarcode() {
    let context = canvas.getContext('2d');
    let imageData = context.getImageData(0, 0, canvas.width, canvas.height);
    let code = callToWasmUnimplemented(imageData.data, imageData.width, imageData.height, {
        inversionAttempts: "dontInvert",
    });
    if (code) {
        alert(code.data);
    }
}


camera_button.addEventListener('click', async function() {
   	let stream = null;

    try {
    	stream = await navigator.mediaDevices.getUserMedia({ video: true, audio: false });
    }
    catch(error) {
    	alert(error.message);
    	return;
    }

    video.srcObject = stream;

    video.style.display = 'block';
    camera_button.style.display = 'none';
    click_button.style.display = 'block';
    // run recognition three times a second.
    setInterval(recognizeBarcode, 300);
});

click_button.addEventListener('click', function() {
    canvas.getContext('2d').drawImage(video, 0, 0, canvas.width, canvas.height);
   	let image_data_url = canvas.toDataURL('image/jpeg');
    
    dataurl.value = image_data_url;
    dataurl_container.style.display = 'block';
});

</script>


</body></html>

hschimke · 2023-01-14T19:51:16Z

very early initial version (0.1.0) available on npm: https://www.npmjs.com/package/rxing-wasm

the detection code for datamatrix behaves oddly when the images provided to it are small, so keep that in mind.

I'd call it an early beta release.

arogozhnikov · 2023-01-15T06:19:41Z

so far no success in detecting anything in js, likely doing something wrong

I've tested on QR codes, in very rare cases it throws "Unreachable", which is probably equal to panic in rust.
In all other cases it just throws an exception "Not Found",

I've set equal dimensions to height and width (480x480) in case there is some mismatch in element ordering, but nothing changed.

I use following functions (checked that output looks sane):

function convertCanvasToGrayscale(canvas) {
    let context = canvas.getContext('2d');
    let height = canvas.height;
    let width = canvas.width;
    let imageData = context.getImageData(0, 0, width, height);

    let data = imageData.data;
    let array = new Uint8Array(data.length / 4);
    // get only the red channel
    for (let i = 0; i < array.length; i += 1) {
        array[i] = data[i * 4];
    }
    return {array, width, height};
}

function convertCanvasToUint32(canvas) {
    let context = canvas.getContext('2d');
    let height = canvas.height;
    let width = canvas.width;
    let imageData = context.getImageData(0, 0, width, height);

    let data = imageData.data;
    console.assert(data.length === width * height * 4);
    let array = new Uint32Array(width * height);
    for (let i = 0; i < array.length; i += 1) {
        // unsure about the ordering
        array[i] = data[i * 4] + (data[i * 4 + 1] << 8) + (data[i * 4 + 2] << 16) + (data[i * 4 + 3] << 24);
    }
    return {array, width, height};
}

outputs are used like this:

        const {array, width, height} = convertCanvasToGrayscale(canvas);
        let parsedBarcode = decode_barcode(array, width, height);

or

        const {array, width, height} = convertCanvasToUint32(canvas);
        let parsedBarcode = decode_barcode_rgb(array, width, height);

arogozhnikov · 2023-01-15T08:06:40Z

encoding into ascii works without issues

hschimke · 2023-01-15T17:00:04Z

I generated the below qrcode and tested it with the rust luma8 decoder. It didn't have any issue decoding it, but I admit to not having tested that as extensively as the image decoder. You might try using it as the input.

Here is the luma conversion that rxing (and zxing) use internally when an image is used as the source:

Luma([((306 * (red as u64)
                    + 601 * (green as u64)
                    + 117 * (blue as u64)
                    + 0x200)
                    >> 10) as u8])

You might try using that as your conversion.

For rgb images the conversion from u32 uses the following bit shifts:

for offset in 0..size {
            //for (int offset = 0; offset < size; offset++) {
            let pixel = pixels[offset];
            let r = (pixel >> 16) & 0xff; // red
            let g2 = (pixel >> 7) & 0x1fe; // 2 * green
            let b = pixel & 0xff; // blue
                                  // Calculate green-favouring average cheaply
            luminances[offset] = ((r + g2 + b) / 4).try_into().unwrap();
        }

There are a lot of weird issues with the datamatrix decoder, most of which I think are present in the original java. It detects datamatrix codes perfectly, but then when it comes to grid sampling them it gives up some strange data. It's the same code used by both the qrcode and aztec components, which handle it fine, so I suspect it's an issue with how the datamatrix decoder handles the input. I'm still poking at it.

If the barcodes are "pure" barcodes, that is, the image contains nothing but a barcode, then the decoder works perfectly (though that requires a bit of extra config for the decoder to know that they're "pure" barcodes).

For a simpler (no wasm) interface you could also look at: https://github.com/zxing-js/library and https://github.com/LazarSoft/jsqrcode

Sample qrcode (verified to decode with luma8) (verified in rust)

Sample raw luma bytes, verified to decode with the luma8 source (80x80 image) (in rust)
luma_raw.txt

hschimke · 2023-01-15T17:20:30Z

I also have a potential fix for datamatrix reading, but still implementing

arogozhnikov · 2023-01-15T19:26:22Z

I've created a sample BW barcode too, it should be readable without doubt - but I get "unreachable" on every call, most likely this is the reason:

rustwasm/wasm-pack#724

hschimke · 2023-01-15T19:57:03Z

Ah, didn't realize SystemTime was a no go. I'll get wasm feature added to Chrono and get system time out.

hschimke · 2023-01-15T22:05:29Z

https://www.npmjs.com/package/rxing-wasm should enable the correct features in chrono crate and remove the SystemTime calls

arogozhnikov · 2023-01-16T11:21:40Z

it works now 🎉 ! Fast enough to run 10 times a second on 1024x768 (did not experiment much with sizes).

Works nicely with QR, finicky with ECC200 DMRE

hschimke · 2023-01-16T15:36:48Z

I'm in the process of porting the c++ datamatrix detector. Apparently it's better in some situations.

hschimke · 2023-01-19T01:06:24Z

I just pushed v0.1.2 of the npm package. After a harrowing trip into c++, rxing now includes the c++ version of the datamatrix detector (https://github.com/zxing-cpp/zxing-cpp). From my initial testing it's much better at finding symbols, especially when those symbols are small, have very slim borders. It seems specifically more likely to work with DMRE codes, but I don't have a ton to test with.

arogozhnikov · 2023-01-19T19:50:36Z

that's great news, thank you!

I am about to test this in a lab, currently experiencing unrelated issues with deployment 😅
Will follow up after I fix things.

arogozhnikov · 2023-01-25T06:24:55Z

Yup rxing does a good job at detecting datamatrix, great work @hschimke!
Will use rectangular barcodes now 🎉 .

hschimke · 2023-01-26T19:58:04Z

I pushed out v0.1.4 of the NPM package, which allows the use of DecodeHints. This might be helpful if you wanted to limit the types of barcodes being searched for. I haven't benchmarked, but my guess would be that it might speed up detection (possibly by a lot).

It also adds a new method to convert the data from a 2dCanvas into luma8, which might be faster in wasm than javascript, but I'm honestly not sure.

arogozhnikov · 2023-01-28T00:30:46Z

great, thanks for letting know. Will test this soon

hschimke · 2023-01-28T00:57:10Z

I have a couple fixes I'm backporting from c++ this weekend too (all datamatrix related). They will all be rolled up in the 0.1.5 realease on npm once I'm done.

arogozhnikov · 2023-01-28T19:35:26Z

integrated both luma conversion and decode hints (I've left only two types: QR and datamatrix).
All works, does not react to other types of barcodes.

Very crude estimate ~ approximately twice lower CPU load judging by profiler peaks.

hschimke · 2023-02-02T03:05:52Z

Watch out for versions 0.1.7 - 0.1.11. A change in some dependencies has caused some havoc I'm still sorting out. I suggest pinning to 0.1.6

hschimke · 2023-02-02T18:13:28Z

This is resolved with npm version 0.1.12, apparently I didn't correctly pin the wasm_bindgen dependency in my deployment environment, so it worked fine in test and then deployed with an update that broke things, oops!

hschimke · 2023-02-15T22:10:21Z

I’m closing this for now, feel free to open a new issue if anything comes up!

hschimke added the enhancement New feature or request label Jan 12, 2023

hschimke self-assigned this Jan 12, 2023

hschimke closed this as completed Feb 15, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

wasm? #1

wasm? #1

arogozhnikov commented Jan 12, 2023

hschimke commented Jan 12, 2023

hschimke commented Jan 13, 2023

hschimke commented Jan 13, 2023

arogozhnikov commented Jan 14, 2023 •

edited

Loading

hschimke commented Jan 14, 2023

arogozhnikov commented Jan 15, 2023 •

edited

Loading

arogozhnikov commented Jan 15, 2023

hschimke commented Jan 15, 2023

hschimke commented Jan 15, 2023

arogozhnikov commented Jan 15, 2023

hschimke commented Jan 15, 2023

hschimke commented Jan 15, 2023

arogozhnikov commented Jan 16, 2023

hschimke commented Jan 16, 2023

hschimke commented Jan 19, 2023

arogozhnikov commented Jan 19, 2023

arogozhnikov commented Jan 25, 2023

hschimke commented Jan 26, 2023 •

edited

Loading

arogozhnikov commented Jan 28, 2023

hschimke commented Jan 28, 2023

arogozhnikov commented Jan 28, 2023 •

edited

Loading

hschimke commented Feb 2, 2023

hschimke commented Feb 2, 2023

hschimke commented Feb 15, 2023

wasm? #1

wasm? #1

Comments

arogozhnikov commented Jan 12, 2023

hschimke commented Jan 12, 2023

hschimke commented Jan 13, 2023

hschimke commented Jan 13, 2023

arogozhnikov commented Jan 14, 2023 • edited Loading

hschimke commented Jan 14, 2023

arogozhnikov commented Jan 15, 2023 • edited Loading

arogozhnikov commented Jan 15, 2023

hschimke commented Jan 15, 2023

hschimke commented Jan 15, 2023

arogozhnikov commented Jan 15, 2023

hschimke commented Jan 15, 2023

hschimke commented Jan 15, 2023

arogozhnikov commented Jan 16, 2023

hschimke commented Jan 16, 2023

hschimke commented Jan 19, 2023

arogozhnikov commented Jan 19, 2023

arogozhnikov commented Jan 25, 2023

hschimke commented Jan 26, 2023 • edited Loading

arogozhnikov commented Jan 28, 2023

hschimke commented Jan 28, 2023

arogozhnikov commented Jan 28, 2023 • edited Loading

hschimke commented Feb 2, 2023

hschimke commented Feb 2, 2023

hschimke commented Feb 15, 2023

arogozhnikov commented Jan 14, 2023 •

edited

Loading

arogozhnikov commented Jan 15, 2023 •

edited

Loading

hschimke commented Jan 26, 2023 •

edited

Loading

arogozhnikov commented Jan 28, 2023 •

edited

Loading