Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added facility to apply a filter function to captured output. #114

Merged
merged 1 commit into from
Aug 2, 2016

Conversation

kevinmcguinness
Copy link
Contributor

Adds a facility to apply a filter function to captured output. This is useful in situations where your experiment code uses a console based progress bar or similar in the text UI, but you would rather not have the observer store the control characters like backspaces and linefeeds in the database. This can be achieved by setting the captured_out_filter on the run or experiment as follows:

from sacred.utils import apply_backspaces_and_linefeeds

ex = Experiment('progress')
ex.captured_out_filter = apply_backspaces_and_linefeeds

It would be possible to also implement this kind of thing by subclassing the observers and filtering the text there, or by changing the observers directly. However, in this case, the user would need to configure all observers. It would also make it harder to implement adding the observer via the command line interface like you can do now.

@coveralls
Copy link

Coverage Status

Coverage increased (+0.3%) to 90.997% when pulling 4f3cca7 on kevinmcguinness:feature/captured-out-filter into 9446a76 on IDSIA:master.

@Qwlouse
Copy link
Collaborator

Qwlouse commented Aug 2, 2016

Hey Kevin,
what a nice PR. You even added an example. Thanks a lot!
I was thinking about adding this feature myself, but I hesitated because I was concerned with the speed of the filter. But by making it optional and configurable this is not much of a problem. And maybe we can someday add a C-extension to do it fast.

I'm merging it now, but it'll need some work for the new 0.7 release since I'm trying to switch to file-descriptor based capturing.

@Qwlouse Qwlouse merged commit 701aa5c into IDSIA:master Aug 2, 2016
@black-puppydog
Copy link

@Qwlouse: would the new capturing be able to catch printf() and/or cout << from C/C++ extensions?

@Qwlouse
Copy link
Collaborator

Qwlouse commented Oct 10, 2016

For the 0.7 release I've tried to switch to a file-descriptor based capturing that would also catch printf and such. Right now, it doesn't yet work smoothly on windows. But I intend on fixing that problem soon. So from the next release on, all outputs to stdout, even from C extensions should be captured, yes.

@black-puppydog
Copy link

black-puppydog commented Oct 11, 2016

Clever /me just found the 0.7 branch and will play around with that a bit :P

Any chance you have a feature branch lying around somewhere? I'm on Linux exclusively anyway, and have a lot of cout happening. I can work around that for now with the logging mechanism of our cluster here, but collecting everything in sacred would be much preferred :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants