getting decoded output

[ingride]

hi.

any suggestions on how to get a decoded output once I have done

let output = model.forward(&input_ids, &token_ids)?;

Am trying to go from a text input, extract input and token_ids, and then decode to a text output. I tried to do some basic convert but didn't get anywhere. if you have an example, I would appreciate it, am kinda struggling to connect the bits.

[ToluClassics]

Hi @ingride ,

Oh when you get the output_ids, you can decode back to text using the model tokenizer

tokenizer.decode(output)

source: https://github.com/huggingface/candle/blob/9bd94c1ffa0ccfd2bbc9526569b8b8a2a3812027/candle-examples/src/token_output_stream.rs#L27

[ingride]

sorry this might be a very dumb question, but the output is a two dimensional tensor - Base Roberta.

are you suggesting to flatten it similarly to here https://github.com/ToluClassics/candle-tutorial/blob/main/tests/test_roberta.rs#L81 ? because decode takes a &[u32] slice as param.

how do i get the output_ids from the two dimensional tensor / the config

[ingride]

@ToluClassics any thoughts on this?