"Received EOS on DATA" frame after around 1 minute running

[ricmalta]

I setup a demo of the Google Cloud Speech API running on Android with AudioRecord to retrieve audio from the microphone.
Its working but after a while (+/- 1 minute) the channel closes by itself.
I basically used the java StreamRecognizeClient example, including the last changes to the gRPC 1.0 and the ManagedChannelBuilder
There is a definition to set a timeout or am I doing something wrong here?

Here is the main code setup:

Setup channel

private static final List<String> OAUTH2_SCOPES = Arrays.asList("https://www.googleapis.com/auth/cloud-platform");

    public static ManagedChannel createChannel(InputStream authorizationFile, String host, int port) throws IOException {

        GoogleCredentials creds = GoogleCredentials.fromStream(authorizationFile);
        creds = creds.createScoped(OAUTH2_SCOPES);
        ManagedChannel channel =
                ManagedChannelBuilder.forAddress(host, port)
                        .intercept(new ClientAuthInterceptor(creds, Executors.newSingleThreadExecutor()))
                        .build();

        return channel;
    }

Recognize setup and loop

public void recognize() throws InterruptedException, IOException {

        try {
            // Build and send a StreamingRecognizeRequest containing the parameters for
            // processing the audio.
            RecognitionConfig config =
                    RecognitionConfig.newBuilder()
                            .setEncoding(RecognitionConfig.AudioEncoding.LINEAR16)
                            .setSampleRate(this.RECORDER_SAMPLERATE)
                            //.setLanguageCode("en-US")
                            .build();

            // Sreaming config
            StreamingRecognitionConfig streamingConfig =
                    StreamingRecognitionConfig.newBuilder()
                            .setConfig(config)
                            .setInterimResults(true)
                            .setSingleUtterance(false)
                            .build();
            // First request
            StreamingRecognizeRequest initial =
                    StreamingRecognizeRequest.newBuilder().setStreamingConfig(streamingConfig).build();

            requestObserver.onNext(initial);

            // Microphone listener and recorder
            recorder = new AudioRecord(MediaRecorder.AudioSource.MIC,
                    this.RECORDER_SAMPLERATE,
                    this.RECORDER_CHANNELS,
                    this.RECORDER_AUDIO_ENCODING,
                    bufferSize);

            recorder.startRecording();

            byte[] buffer = new byte[bufferSize];
            int recordState;

            // loop through the audio samplings
            while ( (recordState = recorder.read(buffer, 0, buffer.length) ) > -1 ) {

                // skip if there is no data
                if( recordState < 0 )
                    continue;

                // create a new recognition request
                StreamingRecognizeRequest request =
                        StreamingRecognizeRequest.newBuilder()
                                .setAudioContent(ByteString.copyFrom(buffer, 0, buffer.length))
                                .build();

                // put it on the works
                requestObserver.onNext(request);
            }

        } catch (RuntimeException e) {
            // Cancel RPC.
            requestObserver.onError(e);
            throw e;
        }
        // Mark the end of requests.
        requestObserver.onCompleted();
    }

The full code is in this repo: https://github.com/Cloudoki/android-google-cloud-speech-api

Need some help here, because I'm not figuring it out!

Thanks

[puneith]

@ricmalta Can you please take a look at https://github.com/GoogleCloudPlatform/android-docs-samples/tree/master/speech/Speech

[ricmalta]

Hi @puneith finally a working example!
I had some issues finding out how to setup a project with the speech API, combing properly the functionality in a android project. This one is exactly I was expecting from the GoogleCloudPlatform team.

This answers all my questions.

Thanks a lot

[puneith]

@ricmalta Can you point to what difficulties you encountered so that we can improve / fix that experience on the README perhaps?

[ricmalta]

The main difficulties I've was in the channel setup.

I was forced to start reading protocol buffers and gRPC documentation to understand the code and then I find out that there was more than one way to setup the authentication and to create the channel itself. Here in this new example (https://github.com/GoogleCloudPlatform/android-docs-samples/tree/master/speech/Speech) I found the 3rd different one.

I understand the Cloud Speech API still in beta, but the authentication seems to be a very final version already, because its mainly handling the gRPC and its 1.0 already. It would be nice to have a good documentation on the channel authentication, creation and handling in a Google Cloud approach.

The API documentation is mainly a description how the endpoints works, there is no actual documentation how we should implement our application using a speech service (for example).

Hope this could help.

[ricmalta]

I ported the example to my application, including the AudioRecorder implementation.
I was successful on the port but I only receive a 400 Bad Request error from the requests. For every audio data sent to the API I receive the error.

How do I debug this?

[ricmalta]

I retrieve the former error when using this dependencies:

    // Google Cloud Speech API dependencies
    compile ('com.google.auth:google-auth-library-credentials:0.4.0')
    compile ('com.google.auth:google-auth-library-oauth2-http:0.4.0') {
        exclude group: 'com.google.guava', module: 'guava-jdk5'
        exclude group: 'com.google.guava', module: 'slice-guava-jdk5'
    }
    compile ('com.google.api.grpc:grpc-google-cloud-speech-v1beta1:0.0.7'){
        exclude group: 'com.google.guava', module: 'guava-jdk5'
    }
    compile 'com.google.guava:guava:19.0'
    compile 'io.grpc:grpc-auth:1.0.0'
    compile 'io.grpc:grpc-okhttp:1.0.0'
    compile 'io.grpc:grpc-netty:1.0.0'
    compile 'io.grpc:grpc-protobuf:1.0.0'
    compile 'io.grpc:grpc-stub:1.0.0'
    compile 'io.netty:netty-tcnative-boringssl-static:1.1.33.Fork22'

Those was the dependencies configuration I reach when trying to port the java example to android.
After I copied the proto files and changed the gradle to support the stubs compilation my app starts working.

I just have a few question I hope you could answer to me:

I suppose I was using a pre-compiled version of the protobuf stubs in my former gradle config (the one I put above), correct?
Its safe to upgrade the version of the gRPC to 1.0?
Being the proto definition files part of my base code, how should I maintain those? There is a repo where I can get the proto definitions per speech API version?

Thanks a log for your help.

[yaraki]

I suppose I was using a pre-compiled version of the protobuf stubs in my former gradle config (the one I put above), correct?

Yes. The dependency of com.google.api.grpc:grpc-google-cloud-speech-v1beta1 is a jar file, containing several Java classes pre-compiled from protobuf definitions.

Its safe to upgrade the version of the gRPC to 1.0?

The current protobuf definitions cannot be compiled with 1.0.

Being the proto definition files part of my base code, how should I maintain those? There is a repo where I can get the proto definitions per speech API version?

There is no clean solution to this yet. I'll check with the API team.