[Enhancement] Set the max length of text for KoboldAI to a bigger value.

[xplshn]

Currently, the Kobold backend can only produce about one paragraph and half of the other, this is because the max_length is not set to be the value of max_context_length when doing the API call.
See: https://koboldai-koboldcpp-tiefighter.hf.space/api#/v1/get_v1_config_max_context_length
For example:

]@ tgpt --provider koboldai "Write a brief introduction to Shakespeare's works - 90~ words"
Write a brief introduction to Shakespeare's works - 90~ words

.
Shakespeare is often considered the greatest playwright in English literature, with many of his works still being widely performed and studied around the world over
 four centuries after his death. He wrote tragedies (e.g., Romeo and Juliet, Hamlet), comedies (e.g., A Midsummer Night's Dream, Twelfth Night), histories (e.g., Henry
 V, Richard III), and tragicomedies such as Much

]@

Also, note the "." before the response, its been a recurring thing with this backend/provider.

Thanks! Have a great day!

[xplshn]

Also, for streamed text, you can use: https://koboldai-koboldcpp-tiefighter.hf.space/api#/extra/post_extra_generate_stream

[aandrew-me]

I am already using streamed text. And there is no option to set max limit. The one you provided it the max limit of input which is probably 2048 characters. Max limit of output is probably 80 characters - https://koboldai-koboldcpp-tiefighter.hf.space/api#/v1/get_v1_config_max_length

[xplshn]

https://koboldai-koboldcpp-tiefighter.hf.space/api#/v1/get_v1_config_max_length

That is an example, you can get what the current possible max length by pulling from https://koboldai-koboldcpp-tiefighter.hf.space/api#/extra/get_extra_true_max_context_length

[aandrew-me]

https://koboldai-koboldcpp-tiefighter.hf.space/api#/v1/get_v1_config_max_length

That is an example, you can get what the current possible max length by pulling from https://koboldai-koboldcpp-tiefighter.hf.space/api#/extra/get_extra_true_max_context_length

I don't see any option to set length in docs -

{
  "prompt": "Niko the kobold stalked carefully down the alley, his small scaly figure obscured by a dusky cloak that fluttered lightly in the cold winter breeze.",
  "temperature": 0.5,
  "top_p": 0.9
}

But I will try

[xplshn]

This is the example they provide:

{
  "max_context_length": 2048,
  "max_length": 100,
  "prompt": "Niko the kobold stalked carefully down the alley, his small scaly figure obscured by a dusky cloak that fluttered lightly in the cold winter breeze.",
  "quiet": false,
  "rep_pen": 1.1,
  "rep_pen_range": 256,
  "rep_pen_slope": 1,
  "temperature": 0.5,
  "tfs": 1,
  "top_a": 0,
  "top_k": 100,
  "top_p": 0.9,
  "typical": 1
}

For https://koboldai-koboldcpp-tiefighter.hf.space/api#/v1/post_v1_generate

[aandrew-me]

Well the example is not for stream mode, however max length works for stream mode too. I am making the default value 300, later I will add flags for it so that users can adjust it

[xplshn]

Okay, excellent! Thanks for all the hard work!