wip: add pseudo speaker diarization pipeline based on segmentation stitching #201
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
very very cool, I think it also makes more sense with the powerset model because of the bigger window too. Let me know if you need a hand with this! We should also find a more catchy name for pseudo speaker diarization 😃 |
* bump pyannote to 3.0 * add wespeaker inference * add weights normalization, cpu for numpy conversion * unify api * remove try catch * always normalize * use PretrainedSpeakerEmbedding in Loader * Fix min-max normalization equation * fix: remove imports * Change embedding model return type to Callable Co-authored-by: Simon <80467011+sorgfresser@users.noreply.github.com> * fix: remove type checking * remove from active if NaN embeddings * Fix wrong typing of model in `LazyModel` * add docstrings * Simplify EmbeddingModel.__call__() * Add numpy import * add normalize boolean * Update requirements.txt * Update setup.cfg * Apply suggestions from code review * Fix wrong kwarg name * add abstract __call__ * move __call__ to parent class --------- Co-authored-by: Juan Coria <juanmc2005@hotmail.com>
* feat: add support for powerset segmentation models * wip: trying this PowersetAdapter thing * fix: initialize nn.Module before setting attribute * Fix unresolved duration and sample rate * Apply suggestions from code review * fix: remove Inference import * fix: black embedding.py ... though it has nothing to do with this PR... --------- Co-authored-by: Juan Coria <juanmc2005@hotmail.com>
* bump pyannote to 3.0 * add wespeaker inference * add weights normalization, cpu for numpy conversion * unify api * remove try catch * always normalize * use PretrainedSpeakerEmbedding in Loader * Fix min-max normalization equation * fix: remove imports * Change embedding model return type to Callable Co-authored-by: Simon <80467011+sorgfresser@users.noreply.github.com> * fix: remove type checking * remove from active if NaN embeddings * Fix wrong typing of model in `LazyModel` * add docstrings * Simplify EmbeddingModel.__call__() * Add numpy import * add normalize boolean * Update requirements.txt * Update setup.cfg * Apply suggestions from code review * Fix wrong kwarg name * add abstract __call__ * move __call__ to parent class --------- Co-authored-by: Juan Coria <juanmc2005@hotmail.com>
* feat: add support for powerset segmentation models * wip: trying this PowersetAdapter thing * fix: initialize nn.Module before setting attribute * Fix unresolved duration and sample rate * Apply suggestions from code review * fix: remove Inference import * fix: black embedding.py ... though it has nothing to do with this PR... --------- Co-authored-by: Juan Coria <juanmc2005@hotmail.com>
* Add ONNX segmentation and embedding models * Minor readability improvements * Replace onnxruntime with onnxruntime-gpu * ONNX can have only one output * Clean up useless embedding model subclasses * Remove duration and sample_rate properties from SegmentationModel. Clean up code * Update README
* Add initial docs * Include README in docs page * Improve README * Update README * Add docs requirements.txt * Add readthedocs config file * Fix links * Add some docstrings * Ignore private attrs in docs * Add some docstrings. Effectively ignore __init__ * Blacken code * Blacken code with good version * Clean up some code * Fix wrong html title
* Add link to accept conditions of segmentation 3.0 * Add table with available models. Add some latencies * Add more info on selecting different models * Add missing info on available models * Improve top menu * Improve python badge * Move things around. Simplify code and wording * Add dark themed logo * Remove whitespace at the top * Update README.md * Rename from_pyannote to from_pretrained in segmentation and embedding blocks * Separate huggingface links from model name * Fix reproducibility link * Add animated diarization pipeline diagram * Improve pipeline gif * Update README.md * Update snippet gif. Fix torch multiprocessing crash with pyannote 3.1. Other README improvements * Update README.md * Fix bad link
* Update version. Add reproducibility note * Add link to pyannote 3.1 interpolation issue in README
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
As segmentation models are getting better, it might make sense to revisit the idea of stitching based on segmentation alone. That's what this (WIP) pipeline does.
Also, that was an excuse for me to start experimenting with
diartand ... I'm enjoying it so far :)