logging: Ensure slog error level logs don't print stack traces

[francislavoie]

I noticed while testing an error case in cert issuance that sometimes we get stack traces in some logs, which is needlessly noisy (not useful for users, vaguely useful for devs sometimes but we can just enable it as needed while dev-ing).

I realized that zapslog (the zap-slog adapter) sets stack traces to be enabled at ERROR level by default. So we need to set it to a big number to make it not be turned on at all.

The actual problematic one was in certmagic, but I found a couple spots we could adjust in caddy as well. Certmagic PR to follow. caddyserver/certmagic#372

Assistance Disclosure

Used Github Copilot to learn that

[mholt]

Oh, thanks... I kind of like the stack traces though, when people post their logs I know exactly where to look and how the code got there, which is less obvious otherwise.

[francislavoie]

No you don't 😅 it looks like a crash/panic in that case, and it does not match the format of the rest of the logs (it escapes the usual single-line-log format and becomes separately formatted newline delimited content). It's also extremely noisy and unfriendly.

[mholt]

I agree it's not visually pleasing, but it IS useful to know how the code got to that point, which a stack trace shows us.

The newlines are consistent with the formatter though right? Like it's not producing invalid JSON.

Can we at least put a comment next to the lines setting the level to 127? It's unclear why without reading this PR.

Thanks!