Dashboard improvements tracker

[anijain2305]

• Clean Inductor complication cache using Clean/monitor triton cache during benchmarking #1207. Basically, add the flag in the performance runs (look at generated run.py to confirm)
• Flag models with (1) < 0.95x speedup, (2) > 2min compile time, (3) < 0.9x compression ration
• Optional (as this requires previous day run) - Flag models that were passing yesterday but failed today.
• slurm - we run Dashboard on slurm cluster. Currently, we are not able to pipe stdout/stderr from the allocated node to jumphost. This makes debugging terribly hard. If fixed, this will improve QOL dramatically.
• Upload nightly run logs to /fsx before parsing - for backing up parsing failures.

[soumith]

slurm - we run Dashboard on slurm cluster. Currently, we are not able to pipe stdout/stderr from the allocated node to jumphost. This makes debugging terribly hard. If fixed, this will improve QOL dramatically.

You can log stdout/stderr to a file, and then have a tail -f [filename] right? or is this not sufficient?