Doesn't work in electron (or with fallback set to true)

[sebakerckhof]

Environment details

• OS: Debian 10
• Node.js version: 12.4
• npm version: 6.13.4
• @google-cloud/speech version: 3.6.0

Steps to reproduce

1. When you try to create a client in an electron browser window (or with fallback option set to true):

new speech.v1.SpeechClient();

it throws following error:

TypeError [ERR_INVALID_ARG_TYPE]: The "path" argument must be of type string. Received type object
    at validateString (internal/validators.js:112)
    at Object.basename (path.js:1157)
    at GrpcClient.loadProto (/data/git/purpleFres…ild/src/grpc.js:117)
    at new SpeechClient (/data/git/purpleFres…peech_client.js:106)

This happens because:

First there is this check setting the fallback option to true in a browser window:
https://github.com/googleapis/nodejs-speech/blob/master/src/v1/speech_client.ts#L94-L96

Next there is this line loading the proto's:
https://github.com/googleapis/nodejs-speech/blob/master/src/v1/speech_client.ts#L135-L137

So in the case fallback is true, it will load the json file and pass that to loadProto. But loadProto expects a filepath. So I think the require in this line should just not be there:
https://github.com/googleapis/nodejs-speech/blob/master/src/v1/speech_client.ts#L136

However, this is not yet sufficient, since it then will still fail in electron environments. isBrowser should just be false for electron.

[bcoe]

@alexander-fenster potentially worth creating a tracking bug in gax for?

[alexander-fenster]

@sebakerckhof For browser / electron users, fallback should be set to true (automatically depending on existence on window object, or manually by passing {fallback: true} as a constructor option).

const client = new speech.SpeechClient({fallback: true});

If fallback is true, it goes to a different implementation of loadProto that accepts JSON object (the result of require). Since your stack points to src/grpc.ts, this means that fallback was set to false. The stack should only go to src/fallback.ts if fallback is enabled. Can you please set fallback to true in the constructor parameter and post the stack if you still get the error?

Also, make sure you take care of authentication properly, in non-Node.js context you might need to use OAuth2. More details here.

[gormonn]

I have the same issue. And I noticed one weirdness.
google-gax version in repo is:

nodejs-speech/package.json

Line 47 in bec2561

"google-gax": "^2.1.0",

google-gax version in module via yarn add @google-cloud/speech is:
1.11.2
so... v2 have breaking changes, but for some reason, the wrong version is indicated in the package.json npm module ...

I tried to install the package directly from a GIT using:
yarn add https://github.com/googleapis/nodejs-speech

But apparently, due to WinOs, I get an error:
" cp "is not internal or external command, executable program or batch file.

[alexander-fenster]

@gormonn A few problems listed here, let's go one by one :)

1. The main breaking change in google-gax is that we dropped Node 8 support. We'll do it here as well - you can see that the package.json in master already points to google-gax v2, it's just not released yet.

2. Our packages work on Windows but cannot be developed on Windows (without *nix tools installed). When you install from GitHub, you need to be able to run compilation, and it requires *nix tools. If you have Cygwin tools in your path, it will work. It can also be possible to use Ubuntu which is the part of Windows 10, even though I personally did not try that.

3. For the main problem (Electron usage), please pass {fallback: true} to the client constructor. If it fails, I'd appreciate a stack :) Thank you!

[gormonn]

@alexander-fenster Tnx for advice. I'm downloading cygwin and packages now.

While it is downloading, I want to clarify a couple of points. In the Electron Main process (NodeJs v12), the module works without problems. When I try to use this module in the Renderer process (Frontend - browser), I get the main problem.

I watched the call stack. And I found that fallback auto-defined as true, because const client = new speech.SpeechClient() are called on the Electron's browser (Chromium).

I tried to do as you said:
const client = new speech.SpeechClient({fallback: true}), but this makes no sense, for the above reasons.

[alexander-fenster]

Oh, I think I figured out what's wrong. Apparently, Electron does not support browser field, which we rely on in this code in the client:

nodejs-speech/src/v1/speech_client.ts

Lines 101 to 108 in 4abdb5c

	const isBrowser = typeof window !== 'undefined';
	if (isBrowser) {
	opts.fallback = true;
	}
	// If we are in browser, we are already using fallback because of the
	// "browser" field in package.json.
	// But if we were explicitly requested to use fallback, let's do it now.
	this._gaxModule = !isBrowser && opts.fallback ? gax.fallback : gax;

The real browser would've followed the "browser": field in package.json of google-gax module here and jump into fallback scenario automatically.

This indeed looks like a bug, I'll think how to fix it properly and will update this issue.

As a workaround, can you try packing the code with webpack before passing it to Electron? webpack supports "browser" field and should bring you to the fallback code (the one that goes to build/src/fallback.ts in google-gax, not build/src/grpc.ts).

[gormonn]

I make it by browserify app.js -o voice-bundle.js and it works correctly. Thanks a lot!
Now i have other issue? with OAuth2. And I have certain thoughts on this.

Tell me, do I understand the reasons for using OAuth2 on the client side correctly?
This is necessary for the security of our credentials. Correctly?

We have a slightly different principle of software delivery.
We use Electron on interactive terminals for various areas that we manufacture and service on our own.
Therefore, OAuth2 looks like an over-complication of both the user interface and the functionality.

For example:
Let's say we integrate OAuth2 authorization.
In this case, we must give out the account information to our technical specialists. So that when starting the terminal, they entered this data. This already sounds at least unsafe.

Is there a way around this limitation? Using our client-side credentials.

[alexander-fenster]

@gormonn In your specific case (kiosk application), you can use API keys for authentication. We don't advertise this option too much because we really don't want it to be used in browsers (where users could easily see the api key in the source code).

For those who read it: the example below can only be used in kiosk usecase. For regular browsers, please use OAuth2Client to implement a proper authentication workflow.

Quick'n'dirty example: https://gist.github.com/alexander-fenster/5ee5f04423ede10f008ae59e4b10d9c4 (I tested this in Chrome but not in Electron, but should be pretty much the same)
More on API keys: https://cloud.google.com/docs/authentication/api-keys

Let me know if you need help!

[bcoe]

@sebakerckhof @gormonn closing this, since it looks like @alexander-fenster's approach should work.

Please let us know if you bump in to any issues.

[sdetweil]

apikey approach still failing for me, node 12, electron 6.0.12, same path error

works ok when the code is run from terminal window node xxx.js but not inside electron
cant use webpack

just trying to enable GoogleAssistant conversation (ended=continueConversation) when we have an external speech recognizer inside https://github.com/evancohen/smart-mirror.. the main process spins off reco into background task, with emitted recognized words. i need to ask it to stop, as it does hotword detection. and then turn on replica engine with no hotword detection.. I am using GA in console input mode, so can't switch to audio mode in the middle of the conversation

browserWindow is in kiosk mode

[alexander-fenster]

@sdetweil Webpacking the source code would've helped, but if you're unable to, then please just wait for the fix for this which is coming soon (I have a tracking issue for nodejs-vision here, this one is exactly the same problem - but I'll reopen it just to make sure it's tracked properly).

As a workaround, you can fix the module locally to do

this._gaxModule = gax.fallback;

instead of what is written in the line 108 below.

https://github.com/googleapis/nodejs-speech/blob/master/src/v1/speech_client.ts#L108

[sdetweil]

I don't see that file
I see
@google-cloud/speech/build/src/speech/v1p1beta1/speech_client.js
line 81, temp fixed below.. new speechClient works with apikey, but not projectid/keyfile

// If we are in browser, we are already using fallback because of the
// "browser" field in package.json.
// But if we were explicitly requested to use fallback, let's do it now.
const gaxModule = gax.fallback; // !isBrowser && opts.fallback ? gax.fallback : gax;
// Create a gaxGrpc object, with any grpc-specific options
// sent to the client.

well, it got closer

fallback.ts:417 Uncaught Error: The gRPC-fallback client library (e.g. browser version of the library) currently does not support streaming calls.
    at fallback.ts:417
    at Object.SpeechClient._innerApiCalls.<computed> [as streamingRecognize] (speech_client.ts:234)
    at SpeechClient._streamingRecognize (speech_client.ts:372)
    at SpeechClient.streamingRecognize (helpers.ts:73)
    at Object.CloudSpeechRecognizer.startStreaming (/home/sam/sm-new/node_modules/sonus/index.js:43)
    at Object.Sonus.start (/home/sam/sm-new/node_modules/sonus/index.js:169)
    at Socket.<anonymous> (/home/sam/sm-new/node_modules/recorder/index.js:60)
    at Socket.Emitter.emit (/home/sam/sm-new/node_modules/component-emitter/index.js:133)
    at Socket.onevent (/home/sam/sm-new/node_modules/socket.io-client/lib/socket.js:278)
    at Socket.onpacket (/home/sam/sm-new/node_modules/socket.io-client/lib/socket.js:236)
    at Manager.<anonymous> (/home/sam/sm-new/node_modules/component-bind/index.js:21)
    at Manager.Emitter.emit (/home/sam/sm-new/node_modules/component-emitter/index.js:133)
    at Manager.ondecoded (/home/sam/sm-new/node_modules/socket.io-client/lib/manager.js:345)
    at Decoder.<anonymous> (/home/sam/sm-new/node_modules/component-bind/index.js:21)
    at Decoder.Emitter.emit (/home/sam/sm-new/node_modules/component-emitter/index.js:133)
    at Decoder.add (/home/sam/sm-new/node_modules/socket.io-client/node_modules/socket.io-parser/index.js:251)

[sdetweil]

I decided that I would spin off a background node process to handle this until such time as in browser support comes along..

minor change to my package

[alexander-fenster]

@sdetweil If you can move the logic to the background process, this is the best possible solution here.

Two more points:

1. A user has just suggested a workaround in the Vision issue: ImageAnnotatorClient does not work inside Electron.js nodejs-vision#573 (comment) - you can use it until the fix is released.

2. Streaming calls will not work in the fallback. For speech, there is a regular recognize and longRunningRecognize (both will work), but streamingRecognize will not.

[sdetweil]

2, thanks the code is not mine that does the speech reco..

spun off to background task

[rzfzr]

2, thanks the code is not mine that does the speech reco..

spun off to background task

Hello there,
have you managed to do it, do you have an example?
the api call should be inside background.js and I could call it from a vue view?