Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Quagga with Cordova Android app #94

Closed
ant-1 opened this issue Mar 15, 2016 · 16 comments
Closed

Quagga with Cordova Android app #94

ant-1 opened this issue Mar 15, 2016 · 16 comments

Comments

@ant-1
Copy link

ant-1 commented Mar 15, 2016

Hello !

I'm trying to include the quagga barcode scanner in a Cordova Android webapp and I have some troubles.

If I run my webapp in a browser on my pc, everything seems to be fine, my webcam is activated and I can display the livestream (except some errors in the browser log)
But if I run my app on my Android phone as an Android webapp (wrapped in java code), I can't see anything.

The only difference between browser and Android version is the DOM video element created:

In Chrome browser:
<div id="barcodeScanner" style="width: 300px; height: 300px"><video autoplay="true" preload="auto" src="blob:http%3A//localhost%3A3000/916803b0-94b7-4222-a987-da251a332e87"></video><canvas class="drawingBuffer" width="640" height="480"></canvas><br clear="all"></div>

In Android app:
<div id="barcodeScanner" style="width: 300px; height: 300px"><video autoplay="true" preload="auto" src="blob:file%3A///0bf84747-caac-41e8-97b9-6ef224df8692"></video></div>

No problem with the getUserMedia API, it is available on my Android app.
And I also add access permissions to the camera of my phone

Any suggestion to investigate on my problem ?

Thank you

[Edit] Also tried to load in my android "web" application the camera_example.html. It shows nothing on the screen and I get this html:
<body><video src="blob:file%3A///082ae35f-d89b-4c37-9782-d57562d80084"></video><canvas width="2" height="2"></canvas></body>

@nicolas-rempulski
Copy link

Hi,

I hijack your issue to try to answer it and add another question about Cordova. We use quaggaJS in Cordova environment too and had some issues making it works.

I think your issue is about hybrid application native part.
Even if your WebView is able to handle HTML5 and WebRTC, the native part may not allow the application to have access to the camera. If you try to list available MediaSource from your Cordova application with :

MediaStreamTrack.getSources(function(sources){
        console.log(sources);
      });

You should see only one source, which is audio only.

To have access to Camera from your application, and thus let quaggaJS auto-start the stream, you should ask android for permission to access Camera. You have to add to your AndroidManifest.xml

<uses-permission android:name="android.permission.CAMERA" />

Now, media listing should return 3 streams : audio, environment (back camera), user(not sure on the name on this one, but it's the front camera). quaggaJS should now be able to start.

This permission shouldn't be erased by Cordova upon rebuild, but it will be removed on platform update or add.
There is surely a way to add this permission automatically from Cordova config.xml or via Cordova hooks, but we do it only manually for now.

As for our experience / question :

We made it works in our Cordova app but we hit a bottleneck with performance issues. We tested this a few months ago on mobile devices, the video stream was lagging due to the high processing needed by quagga.
We tried several things with configuration, removing all location and debugs informations, halfsampling, play with workers number, use our own canvas and send Image as Imagestream to quagga... We also tried to go inside quagga to customize some process (that didn't end well ;) )

When we open our own stream via WebRTC, the stream is smooth, I would say "Camera application" like. As soon as we quagga.start() the stream start to be laggy (3-4 FPS).
We also tried to use quagga directly on device browser, to ensure it was not a Cordova problem, and we hit the same issue.

I see that there is now, in master, a frequency configuration allowing us to set the analysis number per second. It is a great solution and workaround for mobile environment and I would like to try it but I have an issue with workers postMessage function that I never encountered nor that I understand :

Uncaught DataCloneError: Failed to execute 'postMessage' on 'Worker': An object could not be cloned.

thrown by

        workerThread.worker.postMessage({
            cmd: 'init',
            size: { x: _inputStream.getWidth(), y: _inputStream.getHeight() },
            imageData: workerThread.imageData,
            config: _config
        }, [workerThread.imageData.buffer]);

I read, on threads about workers, that this error can be generated if a postMessage try to pass a parameter which is an object and this object has functions. I cannot figure which parameter it could be in this case or how I could work around it. Or even if it is really the problem here.

Did anyone had this issue on mobile environment or at all ?

Thanks !

@nicolas-rempulski
Copy link

I got it, I will open another issue but the problem was that I specified a "target" in configuration.

I think the error was thrown because HTMLElement specified in "target" is a complex object that the browser can't clone. If I remove it, and thus the _config passed in postMessage doesn't contain it, it works like a charm.

@ant-1
Copy link
Author

ant-1 commented Apr 6, 2016

Thanks Nicolas for the answer.

Actually, as I said in my first post, I've ever added the right user permissions in my android.manifest to access the camera.
And the result remain the same, I do not see any streams.

I've made a simple cordova app to test the stream here
The index.html file is exactly the same that we can find provided in the Quagga examples folder

Any idea to make it work ?

@fatbearLTG
Copy link

Have you tried your project in a android chrome browser?
Will it show the stream OK?

@suavelizard
Copy link

@ant-1 did you manage to resolve your issue?

@tonno
Copy link

tonno commented Nov 22, 2017

I have this exact same problem. I keep getting a PermissionDeniedError in Quagga.init(). I have tried setting all permissions in Androidmanifest.xml, and also requesting the permissions at runtime, but no luck.

I suspect the reason is that the Chrome browser in recent Androids just considers file:///android_asset/www insecure and refuses to let it use the video stream. Even though file:// is supposed to be "potentially secure" and should be allowed to use the video stream.

I am getting desperate. My code works if I run it from HTTPS server, but when I package it into an apk and install it then it doesn't work.

Is there any way to get around that PermissionDeniedError?

Cheers
Tommi Joutsiniemi

@tonno
Copy link

tonno commented Nov 28, 2017

@ericblade
Copy link
Collaborator

Anyone who's commented on this found a specific answer, or know if this problem is still valid?

@mathiasmoeller
Copy link

Anyone who's commented on this found a specific answer, or know if this problem is still valid?

For me this is still a valid problem

@ericblade
Copy link
Collaborator

If someone can give me a quick "how to ship quagga and cordova to a device" tutorial, i can give it a spin and see what i can find...

@mathiasmoeller
Copy link

The main reason it does not work with iOS is due to a problem in Phonegap regarding iOS.
See here: https://github.com/phonegap/phonegap-plugin-media-stream#quirks
Even with access to the MediaStream you can not attach it to a <video> tag. Which Quagga does. As far as I understand it, it is simply not possible to use Quagga (or any other thing that displays a camera video in a <video> tag) with Phonegap on iOS.

@ericblade
Copy link
Collaborator

ericblade commented Nov 23, 2019

hmm. that's weird how that works in iOS, if I had a useful iOS device, i could maybe figure out a way to structure something to work with it.. i have no concrete idea why it doesn't work on Android, though, but my first guess is "permissions", and probably needing to implement the call in Android that actually allows the WebView to access the camera. (the device will ask for permissions correctly when you install it, but Chrome quite possibly still needs to be given permission on it's own)

@chrisrodriguezkinesso
Copy link

how is this still open, anyone got this working with Cordova?

@ericblade
Copy link
Collaborator

Well, I've forked the library over here https://github.com/ericblade/quagga2 ... and I ship it to Android directly to the browser from a webserver. At some point, I would like to be able to load my app directly to device, but it's not been at all a priority for me. If someone wants to try it with my repo, and provide a framework for loading it to devices, I can try to have a look at it.

@chrisrodriguezkinesso
Copy link

chrisrodriguezkinesso commented Jan 7, 2020

I got it working actually, it's related to permissions, just install Cordova permissions plugin, and wrap with permission to enable the camera.

let permissions = cordova.plugins.permissions; permissions.checkPermission(permissions.CAMERA, (res) => { if (!res.hasPermission) { permissions.requestPermission(permissions.CAMERA, open());

@ericblade
Copy link
Collaborator

... sounds like my guess was right. I guess having some experience with implementing Chrome into embedded systems is useful sometimes :-D

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

8 participants