Optional structured logging for running conda build in CI

[dkomisar]

Checklist

• I added a descriptive title
• I searched open requests and couldn't find a duplicate

What is the idea?

conda build does not offer a way to output information in a way that is easily parsed. Most of the interesting cases involve build failures.

Why is this needed?

• In order to know whether to restart a build due to a possible network failure, the CI system needs to know if a build failed while checking out source code or if it failed while it was actually building.
• Information about what dependencies are being used, or what are missing in failures, can be automatically parsed by the CI system and saved along with logs.
• Attestations (signed metadata containing records of build actions) will need to be generated to get to higher SLSA levels.

Some of this stuff is not available at all, and some could potentially be parsed out of the current logs. Parsing things out of logs is ugly but it is easy technically on our end. But it's very fragile if we do it would coordination.

What should happen?

I propose the following. We create a new option --ci to conda build. This will enable some specific loggers (or something equivalent) that are disabled by default. We can print whatever structured logging we need there, we'll put comments near those lines warning others not to mess with them.

Once this is in place then Sirius can write the actual logging statements, although we'll probably need help finding where to put them since none of us know conda build.

Additional Context

No response

[dholth]

I've enjoyed using this https://www.structlog.org/en/stable/standard-library.html even though I find it intractable to configure. conda has a --json option that is perhaps less structured than structlog but moreso than ad-hoc logging.
conda-build's logging is in severe need of an update. For example, it saves all log messages, suppressing any that occur twice during a run.

[dkomisar]

I've enjoyed using this https://www.structlog.org/en/stable/standard-library.html even though I find it intractable to configure. conda has a --json option that is perhaps less structured than structlog but moreso than ad-hoc logging. conda-build's logging is in severe need of an update. For example, it saves all log messages, suppressing any that occur twice during a run.

I think only conda has --json right? We need it in conda build. Could we reuse anything from conda?

[github-actions]

Hi there, thank you for your contribution!

This issue has been automatically marked as stale because it has not had recent activity. It will be closed automatically if no further activity occurs.

If you would like this issue to remain open please:

1. Verify that you can still reproduce the issue at hand
2. Comment that the issue is still reproducible and include:
   - What OS and version you reproduced the issue on
   - What steps you followed to reproduce the issue

NOTE: If this issue was closed prematurely, please leave a comment.

Thanks!