Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support view/resume experiment from external folder #3870

Merged
merged 9 commits into from
Jul 12, 2021
Merged
Show file tree
Hide file tree
Changes from 5 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
54 changes: 54 additions & 0 deletions docs/en_US/Tutorial/Nnictl.rst
Original file line number Diff line number Diff line change
Expand Up @@ -1061,6 +1061,60 @@ Manage experiment information

nnictl experiment load --path [path] --codeDir [codeDir]


*
**nnictl experiment view**


*
Description

View an nni experiment from external folder.

*
Usage

.. code-block:: bash

nnictl experiment view [OPTIONS]

*
Options

.. list-table::
:header-rows: 1
:widths: auto

* - Name, shorthand
- Required
- Default
- Description
* - --experiment_dir, -e
- True
-
- The folder path of nni experiment. Note: please make sure the folder name is consistent with the experiment id.
* - --port, -p
- False
- 8080
- The port used to start an experiment.
* - --url_prefix, -u
- False
-
- The prefix url of an experiment


*
Examples

..

view an external expeirment


.. code-block:: bash

nnictl experiment view --experiment_dir [path]

:raw-html:`<a name="platform"></a>`

Manage platform information
Expand Down
25 changes: 25 additions & 0 deletions nni/tools/nnictl/launcher.py
Original file line number Diff line number Diff line change
Expand Up @@ -574,3 +574,28 @@ def view_experiment(args):
def resume_experiment(args):
'''resume an experiment'''
manage_stopped_experiment(args, 'resume')

def view_external_experiment(args):
'''view a experiment from external path'''
# validate arguments
if not os.path.exists(args.experiment_dir):
print_error('Folder %s does not exist!' % args.experiment_dir)
exit(1)
if not os.path.isdir(args.experiment_dir):
print_error('Path %s is not folder directory!' % args.experiment_dir)
exit(1)
experiment_id = os.path.basename(args.experiment_dir)
Copy link
Contributor

@liuzhe-lz liuzhe-lz Jun 28, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This might not work for ~/nni/foo/ (ends with slash). Please check.
And since the experiment ID is not included in logDir / experimentWorkingDirectory, I think the argument is a little stange.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

add validation logic.

Copy link
Contributor

@liuzhe-lz liuzhe-lz Jul 12, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nnictl view -e /home/lz/nni-experiments/Su83qgOR/ cannot pass validation.
I think it's frustrating since bash will automatically add the tail space with tab completion.
Suggest use Path(experiment_dir).name to replace os.path.basement.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sure, updated to Path(args.experiment_dir).name.

log_dir = os.path.dirname(args.experiment_dir)

experiment_config = Config(experiment_id, log_dir).get_config()
assert 'trainingService' in experiment_config or 'trainingServicePlatform' in experiment_config
try:
if 'trainingServicePlatform' in experiment_config:
experiment_config['logDir'] = log_dir
launch_experiment(args, experiment_config, 'view', experiment_id, 1)
else:
experiment_config['experimentWorkingDirectory'] = log_dir
launch_experiment(args, experiment_config, 'view', experiment_id, 2)
except Exception as exception:
print_error(exception)
exit(1)
8 changes: 7 additions & 1 deletion nni/tools/nnictl/nnictl.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@
import pkg_resources
from colorama import init
from .common_utils import print_error
from .launcher import create_experiment, resume_experiment, view_experiment
from .launcher import create_experiment, resume_experiment, view_experiment, view_external_experiment
from .updater import update_searchspace, update_concurrency, update_duration, update_trialnum, import_data
from .nnictl_utils import stop_experiment, trial_ls, trial_kill, list_experiment, experiment_status,\
log_trial, experiment_clean, platform_clean, experiment_list, \
Expand Down Expand Up @@ -167,6 +167,12 @@ def parse_args():
parser_load_experiment.add_argument('--searchSpacePath', '-s', required=False, help='the path of search space file for \
loaded experiment, this path contains file name. Default in $codeDir/search_space.json')
parser_load_experiment.set_defaults(func=load_experiment)
#view an NNI experiment
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm a little confused. Where is the old view? As far as I remembered, there has already been a view.

Copy link
Contributor

@liuzhe-lz liuzhe-lz Jun 30, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems there are nnictl experiment view and nnictl view.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's really confusing. Why can't we merge these two?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

unity command into nnictl view

parser_view_experiment = parser_experiment_subparsers.add_parser('view', help='view an experiment from external folder')
parser_view_experiment.add_argument('--experiment_dir', '-e', required=True, help='the full path of nni experiment folder')
parser_view_experiment.add_argument('--url_prefix', '-u', dest='url_prefix', help=' set prefix url')
parser_view_experiment.add_argument('--port', '-p', default=DEFAULT_REST_PORT, dest='port', help='the port of experiment')
parser_view_experiment.set_defaults(func=view_external_experiment)

#parse platform command
parser_platform = subparsers.add_parser('platform', help='get platform information')
Expand Down