Update Precacher to No Longer Fatally Error by Default [2.0]

[sameluch]

Merge Checklist

All boxes should be checked before merging the PR (just tick any boxes which don't apply to this PR)

• The toolchain has been rebuilt successfully (or no changes were made to it)
• The toolchain/worker package manifests are up-to-date
• Any updated packages successfully build (or no packages were changed)
• Packages depending on static components modified in this PR (Golang, *-static subpackages, etc.) have had their Release tag incremented.
• Package tests (%check section) have been verified with RUN_CHECK=y for existing SPEC files, or added to new SPEC files
• All package sources are available
• cgmanifest files are up-to-date and sorted (./cgmanifest.json, ./toolkit/scripts/toolchain/cgmanifest.json, .github/workflows/cgmanifest.json)
• LICENSE-MAP files are up-to-date (./LICENSES-AND-NOTICES/SPECS/data/licenses.json, ./LICENSES-AND-NOTICES/SPECS/LICENSES-MAP.md, ./LICENSES-AND-NOTICES/SPECS/LICENSE-EXCEPTIONS.PHOTON)
• All source files have up-to-date hashes in the *.signatures.json files
• sudo make go-tidy-all and sudo make go-test-coverage pass
• Documentation has been updated to match any changes to the build system
• Ready to merge

---

Summary

What does the PR accomplish, why was it needed?
A PR to mirror #14013
The precacher is not required for a build to complete, even if enabled. This change allows the precacher to fail without causing the build to fail in tandem.
The precacher should not be blocking builds which it is enabled for.

Change Log

• Update precacher to run in new non-fatal mode by default
  • PRECACHER_NON_FATAL set to y by default
• Add option PRECACHER_NON_FATAL to maintain current precacher behavior

Does this affect the toolchain?

NO

Test Methodology

• Add small go snippet to precacher.go to simulate network failures during precacher

	if rand.Intn(100) == 0 {
		return false, fmt.Errorf("test failure: download failed intentionally")
	}

• run test using 2.0-stable toolchain with the following command
  sudo make build-packages REBUILD_TOOLCHAIN=n REBUILD_TOOLS=y SRPM_PACK_LIST=words PACKAGE_REBUILD_LIST=words -j100 PRECACHE=y
• Additionally run the following command to ensure the that fatal errors are still existant:
  sudo make build-packages REBUILD_TOOLCHAIN=n REBUILD_TOOLS=y SRPM_PACK_LIST=words PACKAGE_REBUILD_LIST=words -j100 PRECACHE=y PRECACHER_NON_FATAL=n