Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PyNEST test result parsing confused by NEST output #762

Closed
heplesser opened this issue Jun 21, 2017 · 3 comments
Closed

PyNEST test result parsing confused by NEST output #762

heplesser opened this issue Jun 21, 2017 · 3 comments

Comments

@heplesser
Copy link
Contributor

@heplesser heplesser commented Jun 21, 2017

The code parsing the output generated by nosetests to provide a progress indication during testing is confused by output generated by NEST, since the parser relies on lines of the form TestABC ... ok. If NEST input intervenes before ok (or any other result), no progress will be displayed, confusing the user (see also #677).

We thus should revise output parsing, but that should first happen once we have decided how to test in the future #761.

@gtrensch
Copy link
Collaborator

@gtrensch gtrensch commented Jul 9, 2017

@heplesser After some investigation I found that the NEST stderr output interferes with the nosetests messages. nosetests comes with several plugins, e.g to capture stdout. The capture-plugin is on per default. For stderr a similar plugin (nose-capturestderr) is available but needs to be installed manually: https://pypi.python.org/pypi/nose-capturestderr/1.2 I ran some tests and it works quite well in my local environment. If stderr is the only source of the corrupted nosetests output, it should be easily solvable.

nosetests takes the first comment line of a test for reporting. This also causes a strange output. E.g.
The models should behave as iaf_cond_* if a == 0., b == 0. and ... ok

@heplesser
Copy link
Contributor Author

@heplesser heplesser commented Jul 31, 2017

@gtrensch Thanks for investigating! I would not want to make the correct operation of the NEST test suite dependent on the installation of an additional Pyton package. But that package is only a single file of code that appears to be quite directly based on similar code in nose. Could we integrate this capture technique directly in NEST, so that we can capture stderr without the need for installing additional third-party software?

I will schedule a discussion of #761 for the next Open Developer VC.

@gtrensch
Copy link
Collaborator

@gtrensch gtrensch commented Jul 31, 2017

@heplesser I see two aspects which I would handle seperatly:

  • the bug we want to fix for the 2.12.1 release which is due to missing nosetests functionality (stdout is captured while stderr is not)
  • the question whether we stay with nosetests and perhaps move to nose2 or swich to a different Python test framework

Addressing the question of the test framework we want to use, will very likely also solve the stderr-issue. Depending on the descision we make there might be some additional effort.

However, as you stated, we should carry out this discussion (#761) in the next Open Developer VC !

@heplesser heplesser closed this in f2304c9 Aug 21, 2017
@terhorstd terhorstd moved this from In Progress to Done in Evaluate new Python test frameworks Sep 30, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
3 participants
You can’t perform that action at this time.