caplog: add convenience caplog.message() method?

[The-Compiler]

After adjusting my test codebase for #3579 I noticed I do this a lot:

assert len(caplog.records) == 1
assert caplog.messages[0] == 'foobar'

With a quick GitHub search it looks like I'm not the only one.

What about adding a caplog.message() convenience method which does the len assertion and returns the only message?

cc @wcooley

[blueyed]

Doesn't assert caplog.messages == ['foobar'] work?

[The-Compiler]

Err, yes, it does. Didn't think of that for some reason! 😆 Nevermind me then!

[The-Compiler]

One scenario where this doesn't work is when you don't know the full message string - as an example:

assert len(caplog.records) == 1
message = caplog.messages[0]
assert message.startswith('OSError while reading PDF.js file:')

but probably isn't worth to introduce another helper method for that. Kinda wonder whether I should have some kind of helper class using fnmatch in my testsuite so I can do this instead:

assert caplog.messages == [helpers.glob('OSError while reading PDF.js file: *')]

[wcooley]

One scenario where this doesn't work is when you don't know the full message string
...
but probably isn't worth to introduce another helper method for that. Kinda wonder whether I should have some kind of helper class using fnmatch in my testsuite so I can do this instead:

My first thought was to generalize your helper into a class -- say, a list subclass that adds matching methods but it seemed like I had seen something like that before... Oh yes, _pytest.pytester.LineMatcher.

There are implementation details in LineMatcher that make it not quite suitable to using directly but it would be a starting point for a more generalized class and a more generalized class could be reused as the basis for LineMatcher. (There are things that I would probably do differently in general, like not having "lines" in the method names, implementing __str__ instead of str, matching only a single pattern at a time, returning/yielding the matches (and leaving pass/fail to the caller), etc.)

[ssbarnea]

Can we reopen this as I think we do not have yet a solution for checking that "one of the log messages starts with a prefix", something that is very common, especially with tools that append errors received from other systems to the logging line.

At this moment, the only way I see is to loop among each message and manually match it, that is a lot of code for a simple test.

Practical example, checking for: "Failed to locate command: [Errno 2] No such file or directory: 'git'")). The second part of the error does happen to vary slightly between different platforms, so a correct test should only look for the first part.

[nicoddemus]

Reopening as requested. 👍

@ssbarnea what API exactly do you have in mind?

Personally I don't mind adding small helpers like this if they are self-contained and easy to maintain, even if later we grow a more general mechanism as mentioned in #4223 (comment): the simple helper is trivial to add and maintain, while the more general mechanism usually involves tons of discussion and might take years. And once the general mechanism is in-place, we can use it in the implementation of the small utility.

[ssbarnea]

Thanks for reopening. I am not sure yet but I can give an example on how we ended implementing the check at https://github.com/ansible/ansible-lint/blob/8673e5b27a55b88e0752338403045e9ac1cb2ebd/test/TestUtils.py#L161-L188

I think it would be reasonable to assume that during a test the user may want to assure that a list of log messages are created, some of them being only prefixes. If we can find an easier way to implement it, the better. Maybe matching with regex?

Even if we decide that a helper would be too much, it would be great to update the documentation of caplog to include an example covering this.