Implement radio ingestion #8

c-w · 2017-06-06T13:59:01Z

This PR implements Audio Streaming Integration by integrating the SpeechToText-WebSockets-Java with our Spark platform.

The integration works as follows:

We create a transcriber object via the speech-to-text library.
We open a standard URL connection to the radio stream and connect it to the transcriber.
The transcriber takes care of reading the audio stream, converting it to text, etc.
Whenever a transcription is complete, the transcriber calls our callback. In this callback, we store the transcribed audio in our Spark receiver which triggers the downstream Spark Streaming pipeline.

Pipeline in action:

Note that the speech-to-text library is still fairly brittle and will need some improvements to be fully production-ready. I recommend to start playing with the integration, collecting some issues and then fixing them in bulk.

The feature service seems to be broken/changed recently, so to test the core of this PR, I also needed to make our feature service client more robust to failures. These changes are also included in this PR: we now ignore bad responses from the feature service and we also ignore when we're unable to connect to the feature service. Even if the feature service is down, the rest of the pipeline can still work (just the locations-from-text inference piece won't work) so I assert that it's good to make our pipeline robust to feature service outages.

Closes CatalystCode/project-fortis#45

kevinhartman

LGTM

kevinhartman · 2017-06-06T15:08:57Z

src/main/scala/DemoFortis.scala

+            // sentiment detection
+            val text = analyzedRadio.originalItem.text
+            val language = analyzedRadio.analysis.language.getOrElse("")
+            val inferredSentiment = sentimentDetection.detectSentiment(text, language).map(List(_)).getOrElse(List())


toList on Option[String] should do what you want :)

Ugh, Scala strikes again -- that's not what I'd call more readable/understandable. getOrElse seems more explicit to me.

Heh yeah but I think the major offense is just that you can enumerate an Option. Since we're already doing that, toList doesn't really push the boundaries too much further IMO :)

Currently we use getOrElse in many places and isDefined / get in others. We should in bulk move all of this to whichever approach you think is better going forward (e.g. match / case).

kevinhartman · 2017-06-06T15:40:41Z

...in/scala/com/microsoft/partnercatalyst/fortis/spark/streamfactories/RadioStreamFactory.scala

+  private var audioStream: InputStream = _
+  private var transcriber: Transcriber = _
+
+  private lazy val onTranscription = new Func[String] {


Perhaps a comment for another repo, but why is there a custom class for Func coming from speechtotext.utils?

speechtotext.utils.Func is part of the Java speech-to-text websocket package. Java doesn't have a native function interface so I had to make one.

I believe this is intended to fill that gap:
https://docs.oracle.com/javase/8/docs/api/java/util/function/Consumer.html

It shows that most of my Java-days were pre-8. I'll update the STT library and push a new version.

Done in 23d6409.

kevinhartman · 2017-06-06T15:47:49Z

...in/scala/com/microsoft/partnercatalyst/fortis/spark/streamfactories/RadioStreamFactory.scala

+  }
+}
+
+case class RadioTranscription(


I'd move this out to a dto folder. I can imagine clients that may want to work with RadioTranscriptions but not RadioStreamFactory.

Done in 33347bb

kevinhartman · 2017-06-06T15:56:22Z

...in/scala/com/microsoft/partnercatalyst/fortis/spark/streamfactories/RadioStreamFactory.scala

+  }
+}
+
+class RadioInputDStream(


I think we should move this class and its dependencies into a separate package for cleaner separation.

Done in 33347bb

kevinhartman · 2017-06-06T16:01:49Z

...icrosoft/partnercatalyst/fortis/spark/transforms/locations/client/FeatureServiceClient.scala

+    } catch {
+      case ex: ParseException =>
+        logError(s"Unable to parse feature service response: $response", ex)
+        List()


As a client of this code, I'd like to distinguish between the error case and having no features found. Perhaps this interface would be a good candidate for a return type of Try[T]?

Here's a pretty good article describing functional error handling with Try in Scala.

Done in c110efa.

kevinhartman · 2017-06-06T16:06:41Z

...icrosoft/partnercatalyst/fortis/spark/transforms/locations/client/FeatureServiceClient.scala

+    fetchResponse(fetch)
+  }
+
+  private def fetchResponse(url: String): String = {


This looks like another good candidate for Try[String].

Done in c110efa.

kevinhartman · 2017-06-06T17:59:45Z

...icrosoft/partnercatalyst/fortis/spark/transforms/locations/client/FeatureServiceClient.scala

  }

-  private def parseResponse(response: String): Iterable[FeatureServiceFeature] = {
+  private def unpack(responseBody: Try[String], endpointName: String): Iterable[FeatureServiceFeature] = {


The error is still swallowed here. I would expect that clients of the public methods of this class should be able to distinguish between empty results and an internal error.

Right now is not the time to change the public API of this class. We currently have many cases where we're not handling dependency failure so we need to audit this anyways in the future.

c-w · 2017-06-06T18:55:34Z

Addressed all actionable comments from Kevin, now waiting for sign off from @erikschlegel.

erikschlegel

Code looks great @c-w . Only recommendation was to include one other radio stream to prove out that use-case.

Great work!

erikschlegel · 2017-06-07T00:29:38Z

src/main/scala/DemoFortis.scala

@@ -278,6 +314,19 @@ object DemoFortis {
          )
        )
      ),
+      "radio" -> List(
+        ConnectorConfig(
+          "Radio",


Can you provide an example with streaming from more than one source, just to prove out that we can support a list of audio sources.

The radio stream is designed to process a single radio feed. For ingesting multiple radio feeds, we should use multiple radio streams. I'm syncing up with @kevinhartman to see how we'd represent that in the current stream-factory paradigm since your ask is really a question for the stream-factories rather than the stream implementations.

c-w · 2017-06-07T13:08:04Z

Closing this off. Let's revisit the questions around multiple copies of the same stream configured with different parameters as a separate thread.

c-w added 9 commits June 6, 2017 13:43

Add radio ingestion via speech-to-text

4f814c6

Closes CatalystCode/project-fortis#45

Handle feature service failure gracefully

980491f

Fix serialization issue

ca0cdc8

Ensure radio stream is discoverable

c8dc543

Handle feature service http failure gracefully

65f49f3

Add audio conversion libraries

419975b

Document speech token requirement

d1e9030

Add second radio example in French

91c1297

Merge branch 'master' into radio-ingestion

f356411

c-w requested review from kevinhartman and erikschlegel June 6, 2017 13:59

c-w added the in progress label Jun 6, 2017

kevinhartman approved these changes Jun 6, 2017

View reviewed changes

c-w added 3 commits June 6, 2017 18:44

Refactor to Try

c110efa

Extract radio stream creation code

33347bb

Fix test

d3f82e4

kevinhartman reviewed Jun 6, 2017

View reviewed changes

c-w added 3 commits June 6, 2017 20:35

Replace Func<T> with Consumer<T>

23d6409

Ensure that java.util.function is defined

3de8e4d

Attempt to force jdk8 on Travis

a20c1e4

erikschlegel approved these changes Jun 7, 2017

View reviewed changes

c-w merged commit e6bffa3 into master Jun 7, 2017

c-w removed the in progress label Jun 7, 2017

c-w deleted the radio-ingestion branch June 7, 2017 13:10

Implement radio ingestion #8

Implement radio ingestion #8

c-w commented Jun 6, 2017

kevinhartman left a comment

kevinhartman Jun 6, 2017

c-w Jun 6, 2017 •

edited

kevinhartman Jun 6, 2017

c-w Jun 6, 2017

kevinhartman Jun 6, 2017

kevinhartman Jun 6, 2017

c-w Jun 6, 2017

kevinhartman Jun 6, 2017

c-w Jun 6, 2017 •

edited

c-w Jun 6, 2017

kevinhartman Jun 6, 2017

c-w Jun 6, 2017

kevinhartman Jun 6, 2017

c-w Jun 6, 2017

kevinhartman Jun 6, 2017

c-w Jun 6, 2017

kevinhartman Jun 6, 2017

c-w Jun 6, 2017

kevinhartman Jun 6, 2017

c-w Jun 6, 2017

c-w commented Jun 6, 2017

erikschlegel left a comment

erikschlegel Jun 7, 2017

c-w Jun 7, 2017

c-w commented Jun 7, 2017

Implement radio ingestion #8

Implement radio ingestion #8

Conversation

c-w commented Jun 6, 2017

kevinhartman left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

c-w Jun 6, 2017 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

c-w Jun 6, 2017 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

c-w commented Jun 6, 2017

erikschlegel left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

c-w commented Jun 7, 2017

c-w Jun 6, 2017 •

edited

c-w Jun 6, 2017 •

edited