[Bugfix] fix crash if max_tokens=None

[NikolaBorisov]

Passing max_tokens=None is allowed in the API,
but after recent changes it started crashing.

[NikolaBorisov]

Here is the exception without this patch

  File "/usr/local/lib/python3.10/dist-packages/starlette/routing.py", line 72, in app
    response = await func(request)
  File "/usr/local/lib/python3.10/dist-packages/fastapi/routing.py", line 299, in app
    raise e
  File "/usr/local/lib/python3.10/dist-packages/fastapi/routing.py", line 294, in app
    raw_response = await run_endpoint_function(
  File "/usr/local/lib/python3.10/dist-packages/fastapi/routing.py", line 191, in run_endpoint_function
    return await dependant.call(**values)
  File "/workspace/vllm/entrypoints/openai/api_server.py", line 143, in create_completion
    generator = await openai_serving_completion.create_completion(
  File "/workspace/vllm/entrypoints/openai/serving_completion.py", line 237, in create_completion
    sampling_params = request.to_sampling_params()
  File "/workspace/vllm/entrypoints/openai/protocol.py", line 134, in to_sampling_params
    return SamplingParams(
  File "/workspace/vllm/sampling_params.py", line 148, in __init__
    self._verify_args()
  File "/workspace/vllm/sampling_params.py", line 186, in _verify_args
    if self.max_tokens < 1:
TypeError: '<' not supported between instances of 'NoneType' and 'int'

[zhuohan123]

If possible, can you add a small test in test_regression so that we can make sure this is correct in the future?

[NikolaBorisov]

If possible, can you add a small test in test_regression so that we can make sure this is correct in the future?

Just did. I also added a new test_sampler_parameters file, where I think this makes more sense. Let me know if you want to keep both or remove one. I don't know how to add this test to the buildkite...

[zhuohan123]

LGTM! Thank you for your contribution!