Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Direct Line Speech support #2621

Merged
merged 47 commits into from Dec 4, 2019
Merged

Add Direct Line Speech support #2621

merged 47 commits into from Dec 4, 2019

Conversation

@compulim
Copy link
Collaborator

compulim commented Nov 21, 2019

Fixes #2571. Fixes #2572. Fixes #2574.

Changelog Entry

Added

Description

  • Add Direct Line Speech SDK package
  • Bundle Direct Line Speech SDK with Web Chat CDN bundle and NPM package
  • Add new sample 06.i.directline-speech-sample

Specific Changes

  • Added a new package botframework-directlinespeech-sdk under the folder /packages/directlinespeech-sdk
    • This is the Direct Line Speech SDK adapter set, including both DirectLineJS-alt, and Web Speech API ponyfill
  • Support unabortable speech recognition engine
    • If the speech recognition engine does not allow abort, we will disable microphone button while recognition is ongoing
  • We will start sending detailed speech recognition result to the bot, including confidence and actual transcript
  • We will peacefully ignore OAuth if the DLJS adapter does not support it
  • New 06.i.direct-line-speech sample

  • Add documentation to SPEECH.md
  • Testing Added
    • We need manual testing
Copy link
Member

stevengum left a comment

@compulim, do we need the commits before adbc6e to get DLS integration, or are those separate changes?

I'd like to take a look through this PR, but I'm not familiar with the codebase and what the other commits are needed for 😔

(I'd like to lessen my cognitive load in this PR if possible 😄)

@compulim compulim changed the title Add Direct Line Speech support [DRAFT] Add Direct Line Speech support Nov 21, 2019
@compulim compulim force-pushed the compulim:feat-dlspeech branch from 487c152 to 1eda8f5 Dec 3, 2019
@compulim compulim marked this pull request as ready for review Dec 3, 2019
@compulim compulim force-pushed the compulim:feat-dlspeech branch from 676873b to 5e90a4b Dec 3, 2019
@compulim compulim changed the title [DRAFT] Add Direct Line Speech support Add Direct Line Speech support Dec 3, 2019
@cwhitten

This comment has been minimized.

Copy link
Member

cwhitten commented Dec 3, 2019

The CI error is coming from https://registry.npmjs.org/botframework-directlinespeech-sdk not being found

@corinagum corinagum added the R7 label Dec 3, 2019
Copy link
Member

corinagum left a comment

finished the 06.i readme

serve.json Outdated Show resolved Hide resolved
samples/06.i.direct-line-speech/README.md Outdated Show resolved Hide resolved
samples/06.i.direct-line-speech/README.md Outdated Show resolved Hide resolved
samples/06.i.direct-line-speech/README.md Outdated Show resolved Hide resolved
samples/06.i.direct-line-speech/README.md Outdated Show resolved Hide resolved
samples/06.i.direct-line-speech/README.md Outdated Show resolved Hide resolved
samples/06.i.direct-line-speech/README.md Outdated Show resolved Hide resolved
samples/06.i.direct-line-speech/README.md Outdated Show resolved Hide resolved
const EMPTY_MP3_BASE64 =
'SUQzBAAAAAAAI1RTU0UAAAAPAAADTGF2ZjU3LjU2LjEwMQAAAAAAAAAAAAAA//tAwAAAAAAAAAAAAAAAAAAAAAAASW5mbwAAAA8AAAACAAABhgC7u7u7u7u7u7u7u7u7u7u7u7u7u7u7u7u7u7u7u7u7u7u7u7u7u7u7u7u7u7u7u7u7//////////////////////////////////////////////////////////////////8AAAAATGF2YzU3LjY0AAAAAAAAAAAAAAAAJAUHAAAAAAAAAYYoRBqpAAAAAAD/+xDEAAPAAAGkAAAAIAAANIAAAARMQU1FMy45OS41VVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVf/7EMQpg8AAAaQAAAAgAAA0gAAABFVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVV';
Comment on lines +4 to +5

This comment has been minimized.

Copy link
@corinagum

This comment has been minimized.

Copy link
@cwhitten

cwhitten Dec 4, 2019

Member

I've never seen this in an application before but it makes sense!

This comment has been minimized.

Copy link
@compulim

compulim Dec 4, 2019

Author Collaborator

😅 I found a better way to prepare white noise programmatically. But need time to prove whether it works. You know, Safari is quite picky on Web Audio and this code already works in our adapter.

Not to mention the new code could be just about same length as this one.

BTW, this one is generated by ffmpeg -f lavfi -i anullsrc=r=44100:cl=mono -t 0.01 -q:a 9 -acodec libmp3lame out.mp3.

@compulim compulim force-pushed the compulim:feat-dlspeech branch from 97672b9 to fb4ad40 Dec 4, 2019
@coveralls

This comment has been minimized.

Copy link

coveralls commented Dec 4, 2019

Coverage Status

Coverage decreased (-2.8%) to 60.848% when pulling fb4ad40 on compulim:feat-dlspeech into 0fcd7fe on microsoft:master.

@compulim compulim merged commit 1f100e7 into microsoft:master Dec 4, 2019
3 of 4 checks passed
3 of 4 checks passed
coverage/coveralls Coverage decreased (-2.8%) to 60.848%
Details
BotFramework-WebChat-CI-PR #93754 succeeded
Details
Build-SSO-for-Enterprise-Apps-CI #20191204.11 succeeded
Details
license/cla All CLA requirements met.
Details
@corinagum corinagum mentioned this pull request Dec 13, 2019
65 of 73 tasks complete
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
5 participants
You can’t perform that action at this time.