You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Describe the bug
command line arguments are not processed correctly when the arguments are passed as shortform instead of full argument name.
Example: if -e is passed instead of --eventlogs . It is failing with different reasons on ascli and spark_rapids_user_tools
Steps/Code to reproduce bug
For ascli:
ascli profiling -e ~/eventlogs-path/profile_testing_logs
Output:
Application Logs:
--------------------
Location: /home/nartal/.spark_rapids_tools/logs/prof_20231123002537_A76aEc59.log
In case of any errors, please share the log file with the Spark RAPIDS team.
2023-11-22 16:25:37,685 ERROR spark_rapids_tools.argparser: Validation err: 1 validation error for ProfileUserArgModel
Cannot run tool cmd. Cannot define eventlogs from input (platform, cluster, and eventlogs)
Error: [type=invalid_argument, input_value=ArgsKwargs((), {'eventlog... 'output_folder': None}), input_type=ArgsKwargs]
NAME
ascli profiling - The Profiling cmd provides information which can be used for debugging and profiling Apache Spark applications running on accelerated GPU cluster.
Above issue is a regression. Seen after PR- #646 got merged.
For spark_rapids_user_tools:
spark_rapids_user_tools onprem profiling -e ~/eventlogs-onprem/profile_testing_logs/databricks_runtime82cpurolledcompressed/
Processing...⡿2023-11-22 16:30:27,792 ERROR rapids.tools.profiling: Eventlogs list is empty. The cluster Spark properties may be missing "spark.eventLog.dir". Re-run the command passing "--eventlogs" flag to the wrapper.
2023-11-22 16:30:27,792 ERROR root: Profiling. Raised an error in phase [Process-Arguments]
Traceback (most recent call last):
cc: @parthosa - Could you take a look.
ascli issue is seen on dev branch. spark_rapids_user_tools issue could be similar to ascli. Please feel free to file a separate issue if both are different.
Expected behavior
command line arguments should be processed correctly
The text was updated successfully, but these errors were encountered:
This is a known bug from the fire plugin. It does not support short flags along with **kwargs. We use **rapids_options to pass additional arguments to the JAR.
We need to come up with an alternative handling of **rapids_options due to this issue.
Describe the bug
command line arguments are not processed correctly when the arguments are passed as shortform instead of full argument name.
Example: if
-e
is passed instead of--eventlogs
. It is failing with different reasons onascli
andspark_rapids_user_tools
Steps/Code to reproduce bug
Above issue is a regression. Seen after PR- #646 got merged.
cc: @parthosa - Could you take a look.
ascli issue is seen on dev branch. spark_rapids_user_tools issue could be similar to ascli. Please feel free to file a separate issue if both are different.
Expected behavior
command line arguments should be processed correctly
The text was updated successfully, but these errors were encountered: