[Draft][CI/Build] Optimize models tests

[DarkLight1337]

The models tests keep getting interrupted (presumably due to running too long). This PR attempts to reduce the running time via:

• Share the HuggingFace cache between Kubernetes containers during CI by storing it in a hostPath volume.
  • Note: hostPath volumes have associated security risks. Is there another way for agent-stack-k8s to use a persistent volume?
• Disabling graph construction (considering that the vLLM model is only run once per test, not including the profile run).

This PR also adds tqdm to the common dependencies. This is not actually a new dependency since it is currently used in vllm/entrypoints/llm.py; I just found that it is not in the requirements.txt file when trying to use tqdm for downloading the models.

[DarkLight1337]

Using eager mode doesn't seem to lead to significant improvement. It seems that the bottleneck is in downloading the models, so we should parallelize this process.

[DarkLight1337]

Tbh it is probably better if we have a way to avoid re-downloading the models each time. Any thoughts?

[DarkLight1337]

I'm not that experienced in Kubernetes but from my understanding, placing the HuggingFace cache inside a Volume should avoid the need to redownload the models when tests are run again in the same Pod.

@rkooo567 is it possible on your end to force the CI to run on the same Pod so we can test whether the cache actually works in this way?

[rkooo567]

Hmm I am not super familiar with how CI works actually (idk if we even use k8s under the hood). cc @simon-mo for thoughts..