fix(buildkite): improve error handling for HTTP requests and add fallback for 404 responses (#1040)

[MattDevy]

BEGIN_COMMIT_OVERRIDE
chore(buildkite): improve error handling for HTTP requests and add fallback for 404 responses (#1040) (#1051)
END_COMMIT_OVERRIDE

[prodsecmachine]

✅ Snyk checks have passed. No issues have been found so far.

Status	Scanner	Critical	High	Medium	Low	Total (0)
✅	Open Source Security	0	0	0	0	0 issues
✅	Licenses	0	0	0	0	0 issues

💻 Catch issues earlier using the plugins for VS Code, JetBrains IDEs, Visual Studio, and Eclipse.