Openai api completion params [seed, temperature, max_tokens, system_fingerprint]

[vmpuri]

Implement [seed, temperature, max_tokens, system_fingerprint(partial)]
Improve determinism of generated responses.

• seed: simply set torch.manual_seed() each time the seed is provided
• max_tokens: default to 16 as per API spec.
• temperature: default to 1.0 as per API spec.
• system_fingerprint: for now, the backend and torch.dtype of the model weights. In the future, we should add model version information to this.

Testing: With the same seeds, the generated response is the same.

Screen.Recording.2024-08-07.at.11.40.10.AM.mov

Tested with the following cURL:

curl http://127.0.0.1:5000/v1/chat \
  -H "Content-Type: application/json" \
  -d '{
    "model": "stories15M",
    "stream": "true",
    "max_tokens": "123",
    "temperature": "0.2",
    "seed": "813",
    "messages": [
      {
        "role": "system",
        "content": "You are a helpful assistant."
      },
      {
        "role": "user",
        "content": "Story about a dog."
      }
    ]
  }'

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/torchchat/1016

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

✅ No Failures

As of commit e08f809 with merge base 67f678b ():
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[byjlw]

several mostly nits other than that looks good