Feature suggestions!

[PotatoSpudowski]

Hi guys lets use this thread for suggesting features!

Current suggestions

• Implement method to get embeddings

[yevhen-kalyna]

As mentioned by me in other tickets:

• Add possibility to choose python version for the module or make it independent from version aka Limited API.
• Build repo to the standalone python package. Import of package should be in python's native way via import.
• Support for package completion.
• Clean up the project's dependencies (Dockerfile should be cleaned up)
• Save/Load state function shouldn't close the script when it has been done.

@PotatoSpudowski What do you think?
Let's discuss this here, and then create issues on need.

[robin-coac]

I could not find any documentation on what parameters our methods support. Example :
fastLlama.Model ?
model.generate ?

[amitsingh19975]

• Commands such as reset, save, load, etc., during chatting.
• Two modes: low memory and performance
• GPU support, if possible
• Documentation

[amitsingh19975]

I could not find any documentation on what parameters our methods support. Example :
fastLlama.Model ?
model.generate ?

We will add documentation once we are done with refactoring.

[PotatoSpudowski]

Hi @yevhen-kalyna

In the new refactor, we are addressing a lot of these issues.

WRT the second point, What we are trying to do is essentially make fastLLaMA something like https://github.com/NVIDIA/FasterTransformer

In the new refactor, we have a folder named 'interfaces' where we will create bindings for multiple popular languages using the bridge.

I am not familiar with building python packages, I would appreciate your feedback on this.
Assume we support JS and Python for now, Would be cool to have packages for these that we can push to pypi and npm. Thoughts?

[stduhpf]

What about an optionnal callback for prompt ingestion to keep track of how many tokens have been ingested and how many are remaining?

[PotatoSpudowski]

@yevhen-kalyna

We have now gotten rid of the python dependency issue! 🕺🏻

Will create issues that you raised so contributors can pick it up!