Add parameter "progress file"

[Jaimecclin]

[please review the Contribution Guidelines prior to submitting your pull request. go ahead and delete this line if you've already reviewed said guidelines.]

What does this PR do?

Add a new parameter to output tqdm progress to a file.

Where should the reviewer start?

tpot/base.py

How should this PR be tested?

1. Test on progress_file as None
2. Test on progress_file as file_handler

Any background context you want to provide?

What are the relevant issues?

[you can link directly to issues by entering # then the number of the issue, for example, #3 links to issue 3]

Screenshots (if appropriate)

Questions:

• Do the docs need to be updated?
  api.md is already updated.
• Does this PR add new (Python) dependencies?
  Nope.

[coveralls]

Coverage increased (+0.004%) to 96.763% when pulling 36c38ab on Jaimecclin:add_progress_file into 6306e60 on EpistasisLab:development.

[weixuanfu]

Thank you for the PR. Could you please provide small examples (with population=20, generations=10) of this log file when verbosity=1, 2, 3?

[weixuanfu]

Could you please rename the progress_file to log_file? I think there are some log infos when verbosity=3.

[Jaimecclin]

Thank you for the PR. Could you please provide small examples (with population=20, generations=10) of this log file when verbosity=1, 2, 3?

Of course I can. I am wondering that I should write an example and take screenshots or write an unit test. How do you think?

[weixuanfu]

unit tests are preferred.

[Jaimecclin]

Hi,
I added a file to test verbosity from 1-3. In case of verbosity as 3, an error was raised. It seems it uses xgboost when verbosity is 3.

Traceback (most recent call last): File "/app/tpot/operator_utils.py", line 80, in source_decode exec('from {} import {}'.format(import_str, op_str)) File "<string>", line 1, in <module> ModuleNotFoundError: No module named 'xgboost'

I am not sure the reason so I just commented out the last unit test. And it also happens in master branch. @weixuanfu Could you please help to answer this? If it's an issue, I can try to figure out it.

[Jaimecclin]

Hi,
I added a file to test verbosity from 1-3. In case of verbosity as 3, an error was raised. It seems it uses xgboost when verbosity is 3.

Traceback (most recent call last): File "/app/tpot/operator_utils.py", line 80, in source_decode exec('from {} import {}'.format(import_str, op_str)) File "<string>", line 1, in <module> ModuleNotFoundError: No module named 'xgboost'

I am not sure the reason so I just commented out the last unit test. And it also happens in master branch. @weixuanfu Could you please help to answer this? If it's an issue, I can try to figure out it.

Hi,

I found the reason why it raise exception only in case of verbosity as 3.

    tmp_path = sourcecode.split('.')
    op_str = tmp_path.pop()
    import_str = '.'.join(tmp_path)
    try:
        if sourcecode.startswith('tpot.'):
            exec('from {} import {}'.format(import_str[4:], op_str))
        else:
            exec('from {} import {}'.format(import_str, op_str))
        op_obj = eval(op_str)
    except Exception as e:
        if verbose > 2:
            raise ImportError('Error: could not import {}.\n{}'.format(sourcecode, e))
        else:
            print('Warning: {} is not available and will not be used by TPOT.'.format(sourcecode))
        op_obj = None

    return import_str, op_str, op_obj

I think it can be modified as printing warning simply instead of raising error. I can put my unit test 3 back as well. Any suggestion is welcome. :)

[weixuanfu]

Thank you for looking into it.

The issue is that there is no xgboost in the test environment. We intentionally let tpot raise that import error when verbosity=3.

I think in the CI envs of TPOT unit tests (Linux and Windows), xgboost should be installed so that this error should not be raised.

Could you please build a similar conda env with xgboost for testing unit test with verbosity=3?

[Jaimecclin]

Ok. I tried it and passed all tests. Please take a look. Thanks.

[weixuanfu]

Thank you for updating. I just found a few minor farther changes. I will merge it to dev branch first for next version of TPOT once those changes are made.

[Jaimecclin]

Hi,
Should I resolve that conflict in base.py?

[weixuanfu]

Hi,
Should I resolve that conflict in base.py?

I have resolved it. Thank you!

[weixuanfu]

Thank you for the PR. I will merge it to dev branch.

[Jaimecclin]

Thanks for your help, @weixuanfu. :)