support text-only input with llama3.2-11b

[Gasoonjia]

This PR enables llama3.2-11b model with text-only input.
Note that this PR is only for CLI pipeline. Will have a separate PR for openai-api update

(torchchat-test) [gasoonjia@server ~/torchchat-32mm (main|REBASE-i|main)]$ python torchchat.py generate llama3.2-11B --prompt "How are you these days"
Using device=cuda NVIDIA PG509-210
Loading model...
Time to load model: 8.88 seconds
-----------------------------------------------------------
How are you these daysI'm just a computer program, so I don't have feelings or emotions like humans do, but thanks for asking! I'm functioning properly and ready to assist with any questions or tasks you may have. How about you? How's your day going?
========================================


      Average tokens/sec (total): 11.66                 
Average tokens/sec (first token): 1.82                 
Average tokens/sec (next tokens): 13.04 

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/torchchat/1216

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 1 New Failure, 1 Cancelled Job

As of commit c0c8033 with merge base ec7b510 ():

NEW FAILURE - The following job has failed:

• pull / runner-et (macos-14-xlarge) (gh)
  RuntimeError: module was compiled against NumPy C-API version 0x10 (NumPy 1.23) but the running NumPy has C-API version 0xe. Check the section C-API incompatibility at the Troubleshooting ImportError section at https://numpy.org/devdocs/user/troubleshooting-importerror.html#c-api-incompatibility for indications on how to solve this problem.

CANCELLED JOB - The following job was cancelled. Please retry:

• pull / runner-et (16-core-ubuntu) (gh)
  ##[error]The operation was canceled.

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[iseeyuan]

What's the difference between tokens and x?

[Gasoonjia]

No difference, all equal to text input
Previously we pull the text input out of batch inside prefill and call it tokens
Now we pull out text input first then forward it into prefill function and call it x
The reason we change the name is other single modality models usually use x to represent the text token inputs.

[joecummings]

You can also check for multimodal content like this so you don't have to check the model type explicitly:

https://github.com/pytorch/torchtune/blob/7da96d18dbddbfcf77e045fe149b1efb6866681d/recipes/dev/generate_v2.py#L123

[Gasoonjia]

@joecummings thanks for your suggestions!
Just curious how do you guys handle the mismatch between input and model requirements? Like when user forward an image input into a single-modality model?

[Jack-Khuu]

The failing test is not relevant. Pushing through