Performance tests ctransformers

[su77ungr]

@hippalectryon-0 introduced HF text embeddings with #45.

May you - if it fits you well - elaborate how this performs?

Edit: missing embeddings port

[hippalectryon-0]

Not sure what the benchmark would be though ?

[su77ungr]

lacking understanding here but this should be seen as a llamacpp port competitor ig... the repo was created a few hours ago so there's still missing a huge chunk. also might support mosaic in the future

[hippalectryon-0]

Implemented in #transfor.

However there are several drawbacks:

• No stop word for now Add support for "stop" in config marella/ctransformers#3 leading to very long generated messages
• No streaming, therefore we don't see anything until the end of the generation

[su77ungr]

streaming added now https://github.com/marella/ctransformers/releases/tag/v0.1.2