error: unexpected argument ‘–max-input-tokens’ found

[moruga123]

System Info

Version: ghcr.io/huggingface/text-generation-inference:1.4

If I add --max-input-tokens 14000 to the docker arguments, it gives this error:

error: unexpected argument ‘–max-input-tokens’ found

When the container is starting, it reports out various arguments that it is using (including defaults), one of these is the prohibitively low default value max_input_length: 1024 .

Information

• Docker
• The CLI directly

Tasks

• An officially supported command
• My own modifications

Reproduction

docker run --gpus all \
    --shm-size 1g \
    -e HUGGING_FACE_HUB_TOKEN=$token \
    -p $myport:80 \
    -v $volume:/data ghcr.io/huggingface/text-generation-inference:1.4 \
    --model-id $model \
    --max-total-tokens 18000 \
    --max-input-tokens 14000

Expected behavior

It should run without errors, since various other CLI commands appear to work fine as docker arguments.

[moruga123]

Using --max-input-length instead. It seems that the newer --max-input-tokens did not yet exist in ghcr.io/huggingface/text-generation-inference:1.4

[github-actions]

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 5 days.