fix: race condition issue

[louis-jan]

Description

chunked_content_provider lambda race condition issue when:

• T1: R1 start -> create run loop lambda and request completion.
  streaming = true | serial queue busy = true.
• T2: R1 done ->
  streaming = false | serial queue busy = false | stop = true (cancel completion request)
• T3: R1 run loop still run ->
  recreate request completion for a completed request? -> zombie request (sus memory leak?).
• T2: R1 completion request cancelled, R2 come -> llama.next_result error -> jump in a loop, R1 back to run loop and create new request completion ->
• T1.5: lost connection -> R1 cancel request in progress -> R2 come & create completion -> single_queue_is_busy true, next_result error -> jump in a loop R3 come, stuck.

1. Its better to not create completion request within lambda, should create before triggering (serially). Apply a serial queue to handle this. Can be easily replaced by Concurrent Queue with workers number = n_parallel later.

This is to make it safer by not triggering 2 lambda functions at the same time and to reduce the state complexity.

2. Embedding request has not handled single_queue_is_busy well, line 456 and 466.
3. Check model loaded failed but still proceed for chat/completion.

Next action:

• Replace Serial Queue = Concurrent Queue with workers number = n_parallel
• Wrap embedding request handler within the shared queue

Issues

• bug: model causing an unknown long wait for a response (and other issues) #430

[tikikun]

LGTM