Search results

Tgi router latency metrics

Status: Open (in progress).

deepinfra/text-generation-inferencenumber 10

#10 In deepinfra/text-generation-inference;

· pathorn opened on Dec 15, 2023

Add support for Mistral

Status: Open (in progress).

deepinfra/text-generation-inferencenumber 9

#9 In deepinfra/text-generation-inference;

· pathorn opened on Dec 7, 2023

· Approved

1

Update flash attention to version v2.3.6

Status: Open (in progress).

deepinfra/text-generation-inferencenumber 8

#8 In deepinfra/text-generation-inference;

· pathorn opened on Dec 1, 2023

Expose raw logit weights to the API if request_logits=true

Status: Open (in progress).

deepinfra/text-generation-inferencenumber 7

#7 In deepinfra/text-generation-inference;

· pathorn opened on Nov 28, 2023

4

Print stats about the KV Cache

Status: Open (in progress).

deepinfra/text-generation-inferencenumber 6

#6 In deepinfra/text-generation-inference;

· NikolaBorisov opened on Oct 10, 2023

Update deps

Status: Open (in progress).

deepinfra/text-generation-inferencenumber 5

#5 In deepinfra/text-generation-inference;

· NikolaBorisov opened on Oct 7, 2023