-
Notifications
You must be signed in to change notification settings - Fork 7
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add basic speech-to-text functionality #33
Conversation
…anguage model processing
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Awesome 🎉
Recording works for me too.
I reviewed most of the files and have to say, that I am very impressed 👍 . Keep up the good work.
Some of my comments are code style-related. It's up to you :).
client/src/pages/Notes.js
Outdated
if (!isRecording) { | ||
setIsRecording(startRecording()); | ||
const socket = getSocket(); | ||
socket.on('recognize', (results) => { | ||
updateNoteContent(results.text); | ||
}); | ||
} else { | ||
setIsRecording(stopRecording().isRecording); | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks like you have a memory leak here.
This event listener is never destroyed.
Every time you start a new recording, it will create a new listener on the recognize
event. This listener (or callback) remains in memory and might be executed.
It's important to remove that listener when you stop recording:
https://socket.io/docs/server-api/#socket-removeListener-eventName-listener
This issue happens in multiple files.
function handleRecognize(recognized) {
updateNoteContent(recognized.text);
}
function handleRecordButtonClick() {
const socket = getSocket();
if (!isRecording) {
setIsRecording(startRecording());
socket.on('recognize', handleRecognize);
} else {
socket.removeListener('recognize', handleRecognize);
setIsRecording(stopRecording().isRecording);
}
}
This would be a good case for an useEffect
if you want to have it more React-like (I recommend this solution):
React.useEffect(() => {
if (!isRecording) {
return;
}
function handleRecognize(recognized) {
updateNoteContent(recognized.text);
}
const socket = getSocket();
socket.on('recognize', handleRecognize);
return () => {
socket.removeListener('recognize', handleRecognize);
}
}, [isRecording]);
function handleRecordButtonClick() {
if (!isRecording) {
setIsRecording(startRecording());
} else {
setIsRecording(stopRecording().isRecording);
}
}
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I applied your solution with useEffect
and it works. Thanks! But I don't know why it works 😅
What is the meaning of this:
return () => {
socket.removeListener('recognize', handleRecognize);
}
To me it looks like this would directly remove the socket.on
event listener. But when I run my application, I see that everything works as intended.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Take a look at the useEffect documentation to understand what the return function is used for :)
Co-Authored-By: Leon Machens <leon.machens@googlemail.com>
…s into recordaudio
Co-Authored-By: Leon Machens <leon.machens@googlemail.com>
Co-Authored-By: Leon Machens <leon.machens@googlemail.com>
Co-Authored-By: Leon Machens <leon.machens@googlemail.com>
…s into recordaudio
In this pull request I added the basic speech-to-text functionality on the website. Users are now able to start and stop recording of notes, speak English text and the app will output transscribed text.
Test it here:
https://dev.deepspeech-notes.haupt.digital/
You have to speak very clearly and a good microphone will also help the accuracy of the language processing.
To create this feature, I developed this process using different technologies:
node-vad
npm module)deepspeech
npm module)socket.io
andsocket.io-client
npm modules)web-voice-processor
npm moduleTo document the functionality, I added this table: