Make test-go.yaml CI check fail when cron jobs `panic`+`recover`

[lucasmrod]

Goal

User story
As a developer,
I want to know when Fleet cron jobs are panicking on CI
so that I can bugs in development/testing instead of in production.

Currently this is not the case because Fleet executes a recover on panic in cron jobs (which we want to keep as is) and CI is not parsing the output to detect this:

fleet/.github/workflows/test-go.yaml

Line 150 in e9b2217

c1grep -A 10 'panic: runtime error: ' /tmp/gotest.log >> /tmp/summary.txt

fleet/server/service/schedule/schedule.go

Line 437 in e9b2217

if r := recover(); r != nil {

Sample of a green CI run with a "hidden" panic:
https://github.com/fleetdm/fleet/actions/runs/10313407643/job/28550120568
which has the following log in Fleet's cron logger output:

{
  "cron": "vulnerabilities",
  "details": "runtime error: invalid memory address or nil pointer dereference",
  "err": "running job",
  "instanceID": "test_instance",
  "jobID": "cron_vulnerabilities",
  "level": "error",
  "schedule": "vulnerabilities"
}

[sharon-fdm]

@lukeheath this is important and estimated to 2 points.
Please help prioritize.

[iansltx]

One corollary to this fix would be to mark panicked/recovered jobs as failed rather than completed. This looks like it might be the place to do that, though that may be a fair amount of scope expansion on this issue (just didn't want it to get lost):

fleet/server/service/schedule/schedule.go

Line 420 in 01014f5

level.Info(s.logger).Log("status", "completed")

See this Slack message from @ksatter

[noahtalerman]

Hey @sharon-fdm do we need to prioritize this before next sprint?

I assume no, so I pulled this off the drafting board. It will go in Luke's queue of engineering initiated stories.

[sharon-fdm]

@noahtalerman , this is not urgent but is important to detect bugs like the critical one we had over the weekend.
Ok to be prioritized by @lukeheath but being estimated to 2 points consider next sprint depending on other tasks.