Whisper.js

The Whisper library is a powerful tool that offers a convenient way to transcribe or translate audio files using Node.js. It leverages the capabilities of the Whisper C++ library, which is responsible for performing the underlying audio processing tasks.

By utilizing native Node.js addons, the Whisper library seamlessly integrates with Node.js applications.

Pre-requisites

• C++ Compiler: You will need a compatible C++ compiler installed on your system. Ensure that you have a C++ compiler installed. Refer to the documentation of your operating system or compiler for installation instructions.

• CMake: CMake is required for building the whisper.cpp library. Make sure you have CMake installed on your system. You can download CMake from the official website: https://cmake.org

Installation

To use the Whisper library, follow these steps:

1. Ensure you have Node.js installed on your machine. You can download it from the official Node.js website: https://nodejs.org

2. Open your terminal or command prompt.

3. Navigate to your project directory.

4. Run the following command to install the Whisper library:

npm install @tech9app/whisper.js

Usage

To transcribe or translate an audio file using the Whisper library, follow these steps:

1. Import the necessary types and the whisper function from the Whisper library:

   import { WhisperParams, SpeechData, whisper } from '@tech9app/whisper.js';

2. Define the parameters for the Whisper process by creating an instance of WhisperParams. This object contains various options and settings for the transcription or translation process. For example:

   const whisperParams: WhisperParams = {
     language: 'en',
     model: '/path/to/models',
     fname_inp: '/path/to/input/file.wav',
     output_txt: true,
   };

3. Call the whisper function, passing the whisperParams as an argument. The whisper function returns a promise that resolves to an array of SpeechData objects or null if the process fails. For example:

   const speechData: SpeechData[] | null = await whisper(whisperParams);

4. Handle the result of the whisper process as needed. The result is an array of SpeechData objects, where each object represents a segment of speech. Each SpeechData object contains the start and end timestamps and the corresponding speech content.

Note: Make sure to handle any potential errors or exceptions during the process.

API Reference

Type Definitions

WhisperParams

An interface representing the parameters for the Whisper process. It includes the following properties:

Parameter	Description	Default Value
n_threads	The number of threads to use for processing	System-dependent
n_processors	The number of processors to use for processing	1
offset_t_ms	The offset time in milliseconds	0
offset_n	The offset value	0
duration_ms	The duration in milliseconds	0
max_context	The maximum context value	-1
max_len	The maximum length	0
best_of	The best of value	5
beam_size	The beam size	-1
word_thold	The word threshold	0.01
entropy_thold	The entropy threshold	2.4
logprob_thold	The log probability threshold	-1.0
speed_up	Whether to enable speed-up optimization	false
translate	Whether to enable translation	false
diarize	Whether to enable diarization	false
output_txt	Whether to output as text	false
output_vtt	Whether to output as VTT	false
output_srt	Whether to output as SRT	false
output_wts	Whether to output as WTS	false
output_csv	Whether to output as CSV	false
print_special	Whether to print special characters	false
print_colors	Whether to print with colors	false
print_progress	Whether to print progress information	false
no_timestamps	Whether to exclude timestamps	false
language	The language to use for processing	undefined
prompt	The initial prompt	undefined
model (required)	The path to the model file	-
fname_inp (required)	The path to the input audio file	-
fname_out	The path to the output file	-

SpeechData

A type representing the speech data of a segment. It includes the following properties:

Field	Description
start	The start timestamp of the segment
end	The end timestamp of the segment
speech	The speech content of the segment

Functions

Whisper

The main function of the Whisper library. It processes audio based on the provided parameters and returns a promise that resolves to an array of SpeechData objects or null.

whisper(params: WhisperParams): Promise<Array<SpeechData> | null>

Examples

Here's an example usage of the Whisper library:

import { WhisperParams, SpeechData, whisper } from '@tech9app/whisper.js';

const whisperParams: WhisperParams = {
  language: 'en',
  model: '/path/to/models',
  fname_inp: '/path/to/input/file.wav',
  output_txt: true,
};

whisper(whisperParams).then((result: Array<SpeechData> | null) => {
  console.log('Result from whisper:', result);
});

In this example, the whisperParams object specifies the parameters for the Whisper process, such as the language, model path, input file path, and output format. The whisper function is called with these parameters, and the resulting promise is handled to obtain the transcribed speech segments.

Download Models

Whisper.js Download Utility

This utility allows you to download a model using Whisper.js.

Usage

The Whisper.js Download Utility supports the following options:

• -m, --modelName <modelName>: Specify the model name.
• -p, --storagePath <storagePath>: Specify the storage path.
• -h, --help: Display help for the command.

To use the Whisper.js Download Utility, make sure you have Node.js installed on your system. Then, open your terminal and run the following command:

npx @tech9app/whisper.js download -m <modelName> -p <storagePath>

Replace <modelName> with the desired model name and <storagePath> with the desired storage path.

List of Models

Model	Disk	RAM
tiny	75 MB	~390 MB
tiny.en	75 MB	~390 MB
base	142 MB	~500 MB
base.en	142 MB	~500 MB
small	466 MB	~1.0 GB
small.en	466 MB	~1.0 GB
medium	1.5 GB	~2.6 GB
medium.en	1.5 GB	~2.6 GB
large-v1	2.9 GB	~4.7 GB
large	2.9 GB	~4.7 GB

Examples

Download the "base.en" model and store it in the "models" directory:

npx @tech9app/whisper.js download -m base.en -p models

License

This library is released under the MIT License. See the LICENSE file for more details.