Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor/new implementation #40

Merged
merged 14 commits into from
Dec 26, 2023
Merged
Show file tree
Hide file tree
Changes from 6 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
130 changes: 94 additions & 36 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,26 +5,32 @@ Speech recognition module for react native using [Vosk](https://github.com/alpha
## Installation

### Library

```sh
npm install -S react-native-vosk
```

### Models

Vosk uses prebuilt models to perform speech recognition offline. You have to download the model(s) that you need on [Vosk official website](https://alphacephei.com/vosk/models)
Avoid using too heavy models, because the computation time required to load them into your app could lead to bad user experience.
Then, unzip the model in your app folder. If you just need to use the iOS version, put the model folder wherever you want, and import it as described below. If you need both iOS and Android to work, you can avoid to copy the model twice for both projects by importing the model from the Android assets folder in XCode. Just do as follow:

### Android
In Android Studio, open the project manager, right-click on your project folder and New > Folder > Assets folder.

In Android Studio, open the project manager, right-click on your project folder and go to `New` > `Folder` > `Assets folder`.

![Android Studio assets folder creation](https://raw.githubusercontent.com/riderodd/react-native-vosk/main/docs/android_studio_assets_folder_creation.png)
kingdcreations marked this conversation as resolved.
Show resolved Hide resolved

Then put the model folder inside the assets folder created. In your file tree it should be located in android\app\src\main\assets. So, if you downloaded the french model named model-fr-fr, you should access the model by going to android\app\src\main\assets\model-fr-fr. In Android studio, your project structure should be like that:
Then put the model folder inside the assets folder created. In your file tree it should be located in `android\app\src\main\assets`. So, if you downloaded the french model named `model-fr-fr`, you should access the model by going to `android\app\src\main\assets\model-fr-fr`. In Android studio, your project structure should be like that:

![Android Studio final project structure](https://raw.githubusercontent.com/riderodd/react-native-vosk/main/docs/android_studio_project_structure.png)

You can import as many models as you want.

### iOS
In XCode, right-click on your project folder, and click on "Add files to [your project name]".

In XCode, right-click on your project folder, and click on `"Add files to [your project name]"`.

![XCode add files to project](https://raw.githubusercontent.com/riderodd/react-native-vosk/main/docs/xcode_add_files_to_folder.png)
kingdcreations marked this conversation as resolved.
Show resolved Hide resolved

Expand All @@ -43,72 +49,124 @@ import Vosk from 'react-native-vosk';

// ...

const voiceRecognition = new Vosk();
const vosk = new Vosk();

vosk
.loadModel('model-en-en')
.then(() => {
const options = {
grammar: ['left', 'right', '[unk]'],
};

voiceRecognition.loadModel('model-en-en').then(() => {
// we can use promise...
const options = ['left', 'right', '[unk]'];
voiceRecognition
vosk
.start(options)
.then((res: string) => {
console.log('Result is: ' + res);
.then(() => {
console.log('Recognizer successfuly started');
})
.catch((e: any) => {
.catch((e) => {
console.log('Error: ' + e);
})
.finally(() => {
console.log("Recognition is complete")
});

// ... or events
const resultEvent = voiceRecognition.onResult((res) => {
console.log('A onResult event has been caught: ' + res.data);
const resultEvent = vosk.onResult((res) => {
console.log('A onResult event has been caught: ' + res);
});

// Don't forget to call resultEvent.remove(); to delete the listener
}).catch(e => {
})
.catch((e) => {
console.error(e);
})

});
```

Note that `start()` method will ask for audio record permission.

[Complete example...](https://github.com/riderodd/react-native-vosk/blob/main/example/src/App.tsx)
[See complete example...](https://github.com/riderodd/react-native-vosk/blob/main/example/src/App.tsx)

### Methods

| Method | Argument | Return | Description |
|---|---|---|---|
| `loadModel` | `path: string` | `Promise` | Loads the voice model used for recognition, it is required before using start method |
| `start` | `grammar: string[]` or `none` | `Promise` | Starts the voice recognition and returns the recognized text as a promised string, you can recognize specific words using the `grammar` argument (ex: ["left", "right"]) according to kaldi's documentation |
| `stop` | `none` | `none` | Stops the recognition |
| `loadModel` | `path: string` | `Promise<void>` | Loads the voice model used for recognition, it is required before using start method. |
| `start` | `options: VoskOptions` or `none` | `Promise<void>` | Starts the recognizer, an `onResult()` event will be fired. |
| `setGrammar` | `grammar: string[]` or `none` | `Promise<void>` | Reconfigures the recognizer to use grammar. |
| `stop` | `none` | `none` | Stops the recognizer. Listener should receive final result if there is any. |
| `unload` | `none` | `none` | Unloads the model, also stops the recognizer. |

### Events
### Types

| VoskOptions | Type | Required | Description |
|---|---|---|---|
| `grammar` | `string[]` | No | Set of phrases the recognizer will seek on which is the closest one from the record, add `"[unk]"` to the set to recognize phrases striclty. |
| `timeout` | `int` | No | Timeout in milliseconds to listen. |

### Events

| Method | Promise return | Description |
|---|---|---|
| `onResult` | The recognized word as a `string` | Triggers on voice recognition result |
| `onFinalResult` | The recognized word as a `string` | Triggers if stopped using `stop()` method |
| `onError` | The error that occured as a `string` or `exception` | Triggers if an error occured |
| `onTimeout` | "timeout" `string` | Triggers on timeout |
| `onPartialResult` | The recognized word as a `string` | Called when partial recognition result is available.|
| `onResult` | The recognized word as a `string` | Called after silence occured. |
| `onFinalResult` | The recognized word as a `string` | Called after stream end, like a `stop()` call |
| `onError` | The error that occured as a `string` or `exception` | Called when an error occurs |
| `onTimeout` | `void` | Called after timeout expired |

### Examples

#### Example
#### Default

```js
const resultEvent = voiceRecognition.onResult((res) => {
console.log('A onResult event has been caught: ' + res.data);
vosk.start().then(() => {
const resultEvent = vosk.onResult((res) => {
console.log('A onResult event has been caught: ' + res);
});
});

resultEvent.remove();

// when done, remember to call resultEvent.remove();
```

#### Using grammar

```js
vosk.start({
grammar: ['left', 'right', '[unk]'],
}).then(() => {
const resultEvent = vosk.onResult((res) => {
if (res === 'left') {
console.log('Go left');
} else if (res === 'right') {
console.log('Go right');
} else {
console.log("Instruction couldn't be recognized");
}
});
});

// when done, remember to call resultEvent.remove();
```

#### Using timeout

```js
vosk.start({
timeout: 5000,
}).then(() => {
const resultEvent = vosk.onResult((res) => {
console.log('An onResult event has been caught: ' + res);
});

const timeoutEvent = vosk.onTimeout(() => {
console.log('Recognizer timed out');
});
})

// when done, remember to clean all listeners;
```

Don't forget to remove the event listener once you don't need it anymore.
#### [Complete example](https://github.com/riderodd/react-native-vosk/blob/main/example/src/App.tsx)

## Contributing

See the [contributing guide](CONTRIBUTING.md) to learn how to contribute to the repository and the development workflow.

## License

MIT
MIT
Loading