Memory error

[TheProtaganist]

Is it possible to host the gpt neo models on a website and make some kind of API, the models are to large to run on my computer. Also It would be nice if to have a stop function so the model knows at what token to stop and be able to add examples of the query needed.

[geekjr]

Which Gpt-Neo are you trying to run? There is a parameter to the Gpt-Neo function called max_length which sets the number of tokens to generate. As far as hosting, you could try to run it in a Google Colab or Paperspace Gradient.

[TheProtaganist]

I was trying to use their biggest model 2.7b. gpt-3 is too expensive and I've been looking for a good alternative to make a chatbot, I haven't found any API that doesn't involve downloading 10GB models to my device. I rather send requests to a website and generate texts from their. My computer space is valuable...

[geekjr]

If this kind of service existed, how much would you be willing to pay?

[TheProtaganist]

I mean I know is a lot of hard work but gpt-neo is a free service I mean everyone has been waiting for a gpt-3 alternative but I mean the price is up to the people making the service.

[geekjr]

If there is enough demand of people who are willing to pay, I can consider it.