Have to push stop twice, once for stopping output and another to stop actual GPU generation, fix

[pseudotensor]

Tried adding click_event twice in cancel, didn't help.

Also, while message stops instantly, generation might continue for 2-3 seconds more since in middle of hard generation.

Also, bit uncontrolled, hits the ValueError when generation finally stopped:

Traceback (most recent call last):
  File "/data/jon/h2o-llm/callbacks.py", line 48, in gentask
    ret = self.mfunc(callback=_callback, **self.kwargs)
  File "/data/jon/h2o-llm/generate.py", line 597, in generate_with_callback
    model.generate(**kwargs)
  File "/home/jon/miniconda3/envs/alpaca/lib/python3.10/site-packages/peft/peft_model.py", line 581, in generate
    outputs = self.base_model.generate(**kwargs)
  File "/home/jon/miniconda3/envs/alpaca/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "/home/jon/miniconda3/envs/alpaca/lib/python3.10/site-packages/transformers/generation/utils.py", line 1406, in generate
    return self.greedy_search(
  File "/home/jon/miniconda3/envs/alpaca/lib/python3.10/site-packages/transformers/generation/utils.py", line 2256, in greedy_search
    if unfinished_sequences.max() == 0 or stopping_criteria(input_ids, scores):
  File "/home/jon/miniconda3/envs/alpaca/lib/python3.10/site-packages/transformers/generation/stopping_criteria.py", line 113, in __call__
    return any(criteria(input_ids, scores) for criteria in self)
  File "/home/jon/miniconda3/envs/alpaca/lib/python3.10/site-packages/transformers/generation/stopping_criteria.py", line 113, in <genexpr>
    return any(criteria(input_ids, scores) for criteria in self)
  File "/data/jon/h2o-llm/callbacks.py", line 22, in __call__
    self.callback_func(input_ids[0])
  File "/data/jon/h2o-llm/callbacks.py", line 43, in _callback
    raise ValueError
ValueError

[pseudotensor]

fixed with newer generation code.

[pseudotensor]

Not entirely fixed. Sometimes generation holds onto thread, and hitting stop doesn't trigger gradio stop, until generation done. Maybe need to use asyncio instead of separate thread for use within gradio.