How export_top_models during experiment.run.

[minsu-enerzai]

Hi!
I am using NNI for multi-trial NAS.
I found that NNI gives top models via export_top_models when it's done.
Is there any function to get the models with top score during experiment.run without using the dash board? (I think it could be possible by extracting info from nni-experiments trials)

[ultmaster]

If you had looked at the implementation details of exp.run(), you will see:

    def run(self, port: int = 8080, wait_completion: bool = True, debug: bool = False) -> bool:
        """
        Run the experiment.

        If wait_completion is True, this function will block until experiment finish or error.

        Return `True` when experiment done; or return `False` when experiment failed.

        Else if wait_completion is False, this function will non-block and return None immediately.
        """
        self.start(port, debug)
        if wait_completion:
            try:
                while True:
                    time.sleep(10)
                    status = self.get_status()
                    if status == 'DONE' or status == 'STOPPED':
                        return True
                    if status == 'ERROR':
                        return False
            except KeyboardInterrupt:
                _logger.warning('KeyboardInterrupt detected')
            finally:
                self.stop()

It's just a start plus a loop of status check. You can implement your own loop and put export top models in the loop. I think that will satisfy your need.

[scarlett2018]

Closing the issue as the question has been answered, feel free to reopen if you have question with the answer.