Llama does not finish...

[HexHands]

I downloaded the Code Llama 7B, Medium. It seems to not finish any text meaning that it never executes the code. Is there any way of fixing this?

[KillianLucas]

Thanks for pointing this out @HexHands!

We should probably have some sort of message that says when it completes because it hits max_tokens (which might be happening here?) . Then we could perhaps ask the user if they want to run the unfinished code block.

Does it do this for very small snippets on your system, like "print hello world"? That will help us determine if it's a max_tokens issue.

[HexHands]

This is the best result I got. It runs the code, but for longer code it doesn't finish it. I would like the ability to be able to set max tokens using code.

[TheWhiteWord]

same problem

[brunoboto96]

Fiy, this was also referenced by me here: #126 (comment)
Even though I've changed the context size on the appropriate files, also had a brief go at tokentrim.py but no luck

[brunoboto96]

#126 (comment) Thanks @jordanbtucker the refactoring now works by editing the params in interpreter.py
Still a bit buggy in between instructions, but at least it does not stop 🚀

[leifktaylor]

This issue looks resolved based on @brunoboto96 comment.

@jordanbtucker