Ensure filestate stack.json file are atomically replaced

[Frassle]

Description

When saving a new version of the stack checkpoint we don't delete the current file, instead we let WriteAll atomically replace it. This ensures that if any concurrent attempts are made to list or read the stack while the write is happening they can still pick up the existing about to be replaced version.

Also inlined the renaming logic to backupObject and so can now log a warning if the backup copy fails.

Fixes #9033

Checklist

• I have added tests that prove my fix is effective or that my feature works

• I have updated the CHANGELOG-PENDING file with my change

• Yes, there are changes in this PR that warrants bumping the Pulumi Service API version

[mikhailshilkov]

This makes sense to me. Probably a larger topic than this change - but should we start adding some test harness around file backends that could cover changes like this one?

[Frassle]

but should we start adding some test harness around file backends that could cover changes like this one?

Yes we should! I did look at if it made sense to write a test for this specific case but it's very very racy so pretty much always passed even without the fix, so I didn't deem it useful enough to keep.