Skip to content

Update Whisper and Argos component README #344

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Dec 21, 2023
Merged

Update Whisper and Argos component README #344

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
4f0da44
Update Whisper and Argos component README.
jrobble Dec 21, 2023
4357c13
Merge branch 'develop' into feat/whisper-readme
jrobble Dec 21, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion python/ArgosTranslation/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Overview

This repository contains source code for the OpenMPF Argos Translation Component.
This repository contains source code for the OpenMPF Argos Translation Component. This component is based on [Argos Translate](https://github.com/argosopentech/argos-translate).

This component translates the input text from a given source language to English. The source language can be provided as a job property, or be indicated in the detection properties from a feed-forward track.

Expand Down
28 changes: 16 additions & 12 deletions python/WhisperSpeechDetection/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,21 +1,24 @@
# Overview

This repository contains source code and model data for the OpenMPF Whisper Speech Detection component.
This component uses the OpenAI Whisper model.
This repository contains source code and model data for the OpenMPF Whisper Speech Detection component. This component
is based on [OpenAI Whisper](https://github.com/openai/whisper).

# Introduction

This component identifies the language spoken in audio and video clips.
This component identifies the language spoken in audio and video clips, can perform speech-to-text on the audio, and can
perform translation directly from the audio.

# Input Properties
- `WHISPER_MODEL_SIZE`: Size of the Whisper model. Whisper has `tiny`, `base`, `small`, `medium`, and `large` models available for multilingual models. English-only models are available in `tiny`, `base`, `small`, and `medium`.
- `WHISPER_MODEL_LANG`: Whisper has English-only models and multilingual models. Set to `en` for English-only models and `multi` for multilingual models.
- `WHISPER_MODE`: Determines whether Whisper will perform language detection, speech-to-text
transcription, or speech translation. If multiple languages are spoken in a single piece of media,
language detection will detect only one of them. English-only models can only transcribe English
audio. Set to `LANGUAGE_DETECTION` for spoken language detection, `TRANSCRIPTION` for
speech-to-text transcription, and `SPEECH_TRANSLATION` for speech translation.
- `AUDIO_LANGUAGE`: Optional property that indicates the language to use for audio translation or transcription. If left as an empty string, Whisper will automatically detect a single language from the first 30 seconds of audio.
- `WHISPER_MODEL_SIZE`: Size of the Whisper model. Whisper has `tiny`, `base`, `small`, `medium`, and `large` models
available for multilingual models. English-only models are available in `tiny`, `base`, `small`, and `medium`.
- `WHISPER_MODEL_LANG`: Whisper has English-only models and multilingual models. Set to `en` for English-only models and
`multi` for multilingual models.
- `WHISPER_MODE`: Determines whether Whisper will perform language detection, speech-to-text transcription, or speech
translation. If multiple languages are spoken in a single piece of media, language detection will detect only one of
them. English-only models can only transcribe English audio. Set to `LANGUAGE_DETECTION` for spoken language
detection, `TRANSCRIPTION` for speech-to-text transcription, and `SPEECH_TRANSLATION` for speech translation.
- `AUDIO_LANGUAGE`: Optional property that indicates the language to use for audio translation or transcription. If left
as an empty string, Whisper will automatically detect a single language from the first 30 seconds of audio.

# Output Properties
- `DETECTED_LANGUAGE`: Language with the highest confidence value.
Expand Down Expand Up @@ -49,7 +52,8 @@ large | English | Correctly translated | Mostly skipped
See [whisper_behavior_notes.md](whisper_behavior_notes.md) for more details.

# Language Identifiers
The following are the ISO 639-1 codes, the ISO 639-3 codes, and their corresponding languages which Whisper can translate to English.
The following are the ISO 639-1 codes, the ISO 639-3 codes, and their corresponding languages which Whisper can
translate to English.

All translations are to English.

Expand Down