fix(metrics): get command relax validation to allow any --metric #334
Conversation
CR2-50 Show custom metrics options in gradient-cli get command
Research: https://linear.app/paperspace/issue/CR2-22/view-custom-metrics-from-cli We are also currently blocking anything that is not in the hard coded list, we dont want to block custom metrics (ie: training_total). If user types a command that doesn't exist, return empty data instead. We are currently showing a hard coded list of options for the get command, we want to add known custom metrics to this list for the help text. |
gradient/cli/experiments.py
Outdated
| default=(constants.BuiltinMetrics.cpu_percentage, constants.BuiltinMetrics.memory_usage), | ||
| help="One or more metrics that you want to read. Defaults to cpuPercentage and memoryUsage", | ||
| help=("One or more metrics that you want to read. Defaults to cpuPercentage and memoryUsage. Custom metrics are {}".format("hello")), # TODO replace "hello" with custom metrics |
There was a problem hiding this comment.
Let's instruct the user to use gradient metrics experiment metrics list to see available custom metrics, and instead we should show the user the contents of constants.METRICS_MAP.
There was a problem hiding this comment.
ohh nice then i'm 5 min away from completing this
robghchen
deleted the
CR2-50-show-custom-metrics-options-in-get-command
branch
|
🎉 This PR is included in version 1.2.4 🎉 The release is available on GitHub release Your semantic-release bot 📦🚀 |
Currently we're only allowing get metrics for this hard coded list of options
cpuPercentage, memoryUsage, gpuMemoryFree, gpuMemoryUsed, gpuPowerDraw, gpuTemp, gpuUtilization, gpuMemoryUtilization. if user inputs anything other than those options, the cli will raise an error. we want to relax this validation to support custom metricsWe also want to show known metrics in the --help text
QA plan:
gradient deployments metrics get --id desmvfs1gxebznb --metric iamcustom, should return null instead of showing an errorgradient deployments metrics --help, check if thegetcommand help text now mentionscpuPercentage, memoryUsage, gpuMemoryFree, gpuMemoryUsed, gpuPowerDraw, gpuTemp, gpuUtilization, gpuMemoryUtilization