docs: add howto configure jubilant logs

[tromai]

This PR is a follow up from:

• feat: improve Jubilant logging mechanism jubilant#351
• feat: add pytest-jubilant logger and drop printing 1000 lines on failure pytest-jubilant#106

This PR add the following content into howto/write-integration-tests-for-a-charm.md:

• How to configure Jubilant log modes from pyproject.toml
• Update the Generate crash dumps section to mention the new pytest-jubilant logger object.

There is one small change in howto/set-up-continuous-integration-for-a-charm.md because it uses the GitHub Action workflow example in howto/write-integration-tests-for-a-charm.md.

[tromai]

I rename to keep it general, as this artifact now contain both the log file generated by pytest, and the files generated by pytest-jubilant.

[dwilding]

I haven't finished reviewing, but need to stop for today. Sending my initial comments in case they are helpful.

[tromai]

Commit d69899c assumes the changes from canonical/pytest-jubilant#106.

[tonyandrewmeyer]

I didn't follow the PRs to adjust the logging earlier, so I might have missed some discussion here, sorry. Do we want to describe these "Brief" and "Verbose" styles here, and do we refer to them by those names anywhere else? It seems like we are just describing how we'd recommend configuring stream and file output, and the names just make it confusing. But if we are using them elsewhere or they have wider meaning, then that would be different.

I think it would be good to have some example output for each of the cases, so people know what kind of content they should get, particularly with different log levels (the format is just standard logging and should be fairly obvious).

In Jubilant or pytest-jubilant logs do we have somewhere information about what gets logged to each level? If so, it would be good to link to that. If not, maybe this is the place for that, although it seems like it belongs in Jubilant.

[tromai]

@tonyandrewmeyer . Thanks for the questions.

Do we want to describe these "Brief" and "Verbose" styles here, and do we refer to them by those names anywhere else?

Brief and Verbose are only defined as names for the modes, and they are not mentioned anywhere else. They originated from the Spec, and followed the CLI standard. I was hoping the names can easily communicate what each mode is. But thinking again, it doesn't give any benefit in this context, and can be confusing. Instead of a name, may be a short description sentence would be good enough.

I think it would be good to have some example output for each of the cases, so people know what kind of content they should get, particularly with different log levels (the format is just standard logging and should be fairly obvious).

Good point. I will add them in. I can reuse the examples in the Spec.

In Jubilant or pytest-jubilant logs do we have somewhere information about what gets logged to each level? If so, it would be good to link to that. If not, maybe this is the place for that, although it seems like it belongs in Jubilant.

The logs don't contain this information.

Or, I assume you are referring to the docs 😅 . We don't have this information anywhere in the docs. Although I'm not sure if we should mention it, since this is not part of our contract, and can change later on ? Ah nevermind, my mistake. I think we definitely want to mention what type of information to expect at each level in Jubilant, and link it here.

[tonyandrewmeyer]

Or, I assume you are referring to the docs 😅.

Yes, sorry!

I think we definitely want to mention what type of information to expect at each level in Jubilant, and link it here.

Agreed. I don't think this makes it an exact promise, and certainly the format is not. But as a user I need some way of deciding what makes it into each level so I can make a sensible choice of level. I think you're right that it should be the type of information rather than specific details.

[dwilding]

Calling Brief, Verbose, and Error "logging modes" could be confusing. As you say, those names aren't used anywhere else. How about trying this structure:

Configure Jubilant logs

Intro as currently

Log levels

Briefly explain the differences between INFO, DEBUG, and ERROR. E.g., how does Jubilant decide what to emit at what level?

Examples

Two examples with minimal discussion.

Here's our recommended config for brief logging:

...

And here's some sample output:

...

Here's a config for verbose logging:

...

And here's some sample output:

...

(I'd drop the ERROR example because I think you'll cover it sufficiently in the "Log levels" section)

Default behaviour

Info as currently

[tromai]

Thank you for the review.

@dwilding I have revised the logging level discussion in the docs following your suggested structure. And I think it works extremely well.

@tonyandrewmeyer I also added a short description on what is shown in each log level by Jubilant. It's under the new Log levels section. Although being minimal, it covers all things that Jubilant is logging. I can work on a more detailed description on the Jubilant side, but I can see that it won't be much different (if we don't go into specific details) 🤔 .

[dwilding]

I added several comments, but all minor. Nice work on this!

[tonyandrewmeyer]

Thanks for this! Nothing much other than agreeing with David's comments.

[dwilding]

I looked over the latest version. Great work!