[Model request] Facebook/BlenderBot for text-generation

[fakerybakery]

Hello,
This project is really cool. I was wondering if it's possible to use the BlenderBot model.
My current code gives me an error:

let pipe = await pipeline('text-generation', model = 'facebook/blenderbot_small-90M')
let out = await pipe("Prompt");
console.log(out);

GET https://huggingface.co/Xenova/transformers.js/resolve/main/quantized/facebook/blenderbot_small-90M/causal-lm-with-past/tokenizer.json 404
Uncaught (in promise) Error: File not found. Could not locate "https://huggingface.co/Xenova/transformers.js/resolve/main/quantized/facebook/blenderbot_small-90M/causal-lm-with-past/tokenizer.json".

I saw that this was added in your Hugging Face repository, but I was wondering what the correct way to implement this is.
Thank you!

[xenova]

Hi! As stated in a similar issue from 2 days ago (#29) I haven't yet implemented BlenderBot (no listed support in the README or on the demo website). I just uploaded the files since I plan to add it (which, I probably shouldn't have done haha). However, I do plan to add support for it in the next few days!

It will need a way to keep track of conversations, but I will look to HF's implementation for inspiration :)

(The error message you got is because I haven't uploaded the tokenizer yet ;) )

[fakerybakery]

Thank you so much!!!

[zap8600]

Hi! As stated in a similar issue from 2 days ago (#29) I haven't yet implemented BlenderBot (no listed support in the README or on the demo website). I just uploaded the files since I plan to add it (which, I probably shouldn't have done haha). However, I do plan to add support for it in the next few days!

It will need a way to keep track of conversations, but I will look to HF's implementation for inspiration :)

(The error message you got is because I haven't uploaded the tokenizer yet ;) )

I believe that the Blenderbot model keeps track of a conversation using certain tokens to tell when a speaker stops and starts talking, like this:
<s>Hello! How are you?</s> <s>I'm good. How are you?</s> <s>I'm doing pretty good.</s>
From what I can tell, this is implemented by the user. My really bad code (in Python) for this is here. Even though this code is for Blenderbot, I believe the same goes for Blenderbot Small. However, there are way better examples than my code. This is just what I've seen in multiple Spaces on Huggingface. I hope this helps!

[xenova]

Thanks @zap8600! I've seen similar examples for blenderbot, but the documentation for blenderbot small does not seem to be as clear. I raised an issue (huggingface/transformers#22301), but it looks like it's been hidden already by other issues.

If possible, do you think you could look into how blenderbot small handles it? (Which is apparently different to normal blenderbot)

[zap8600]

Thanks @zap8600! I've seen similar examples for blenderbot, but the documentation for blenderbot small does not seem to be as clear. I raised an issue (huggingface/transformers#22301), but it looks like it's been hidden already by other issues.

If possible, do you think you could look into how blenderbot small handles it? (Which is apparently different to normal blenderbot)

I've look at the Transformers doc (which is where I based my code on) and it appears to be the same way.

[xenova]

Right, but if you look at the vocabulary of the model, <s> is not one of the tokens.

[zap8600]

Right, but if you look at the vocabulary of the model, <s> is not one of the tokens.

My bad. I've taken a look at the vocabulary file, and the tokens appear to be __start__ and __end__. I haven't tested this yet, but that is what I can tell.

[xenova]

Well, I'll test it once I implement it 😄

One thing I find strange is that there isn't a way to say who is speaking? Is it just assumed by the alternating order of conversations?

[zap8600]

Well, I'll test it once I implement it 😄

One thing I find strange is that there isn't a way to say who is speaking? Is it just assumed by the alternating order of conversations?

No need to test it. It does seem to work. My code has been uploaded to my repo, and you can find it here.

[zap8600]

Yay!

[xenova]

A little late since it wasn't as simple as some of the other models (combined with the fact that the python tokenizer is broken), but it's here now! 🤗 I'll make a release soon, so you'll be able to test it.

[zap8600]

Thank you! I hope my testing code helped some!

[xenova]

It did! Thanks 🤗

[zap8600]

Np! I can't wait to use the BlenderBot model!