Cleaning up --help: Gate Evaluation Arguments

[Jack-Khuu]

The current --help is a mess; it uses a giant add_arguments_for_verb function that doesn't actually filter based on the provided verb subcommand.

This PR is part of a series to clean up this behavior.

Specifically, this PR restricts args that are utilized solely for perplexity evaluation to just the eval subcommand.

• Note this does not do the inverse. Eval still has args that are unrelated to eval. This is addressed later

---

python3 torchchat.py generate --help

usage: torchchat generate [-h] [--chat] [--distributed] [--gui] [--prompt PROMPT] [--is-chat-model] [--seed SEED] [--num-samples NUM_SAMPLES] [--max-new-tokens MAX_NEW_TOKENS] [--top-k TOP_K] [--temperature TEMPERATURE] [--compile]
                          [--compile-prefill] [--sequential-prefill] [--profile PROFILE] [--speculate-k SPECULATE_K] [--draft-checkpoint-path DRAFT_CHECKPOINT_PATH] [--checkpoint-path CHECKPOINT_PATH] [--params-path PARAMS_PATH]
                          [--gguf-path GGUF_PATH] [--tokenizer-path TOKENIZER_PATH] [--output-pte-path OUTPUT_PTE_PATH] [--output-dso-path OUTPUT_DSO_PATH] [--dso-path DSO_PATH] [--pte-path PTE_PATH]
                          [--dtype {fp32,fp16,bf16,float,half,float32,float16,bfloat16,fast,fast16}] [-v] [--quantize QUANTIZE] [--draft-quantize DRAFT_QUANTIZE]
                          [--params-table {13B,70B,CodeLlama-7b-Python-hf,34B,stories42M,30B,stories110M,7B,stories15M,Mistral-7B,Meta-Llama-3-8B}] [--device {fast,cpu,cuda,mps}] [--hf-token HF_TOKEN] [--model-directory MODEL_DIRECTORY]
                          [--port PORT]
                          [model]
...

python3 torchchat.py eval --help

usage: torchchat eval [-h] [--chat] [--distributed] [--gui] [--prompt PROMPT] [--is-chat-model] [--seed SEED] [--num-samples NUM_SAMPLES] [--max-new-tokens MAX_NEW_TOKENS] [--top-k TOP_K] [--temperature TEMPERATURE] [--compile]
                      [--compile-prefill] [--sequential-prefill] [--profile PROFILE] [--speculate-k SPECULATE_K] [--draft-checkpoint-path DRAFT_CHECKPOINT_PATH] [--checkpoint-path CHECKPOINT_PATH] [--params-path PARAMS_PATH]
                      [--gguf-path GGUF_PATH] [--tokenizer-path TOKENIZER_PATH] [--output-pte-path OUTPUT_PTE_PATH] [--output-dso-path OUTPUT_DSO_PATH] [--dso-path DSO_PATH] [--pte-path PTE_PATH]
                      [--dtype {fp32,fp16,bf16,float,half,float32,float16,bfloat16,fast,fast16}] [-v] [--quantize QUANTIZE] [--draft-quantize DRAFT_QUANTIZE]
                      [--params-table {13B,70B,CodeLlama-7b-Python-hf,34B,stories42M,30B,stories110M,7B,stories15M,Mistral-7B,Meta-Llama-3-8B}] [--device {fast,cpu,cuda,mps}] [--tasks TASKS [TASKS ...]] [--limit LIMIT]
                      [--max-seq-length MAX_SEQ_LENGTH] [--hf-token HF_TOKEN] [--model-directory MODEL_DIRECTORY] [--port PORT]
                      [model]

...

---

Notice how tasks, max-seq-length, and limit only show up as args for eval

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/torchchat/885

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 1 New Failure, 1 Cancelled Job

As of commit 6b69d6a with merge base b6b6c1e ():

NEW FAILURE - The following job has failed:

• pull / test-tinystories-executorch (macos-14-xlarge) (gh)
  Process completed with exit code 1.

CANCELLED JOB - The following job was cancelled. Please retry:

• pull / test-tinystories-executorch (16-core-ubuntu) (gh)

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[Jack-Khuu]

Manually merging given pre-existing test failure

Attempting fix in #887