Skip to content

[Caching] Use atomic rename() on non-Windows in FileCacheStorage to avoid partially written cache reads on parallel run - #8426

Open
samsonasik wants to merge 2 commits into
mainfrom
partial-cache
Open

[Caching] Use atomic rename() on non-Windows in FileCacheStorage to avoid partially written cache reads on parallel run#8426
samsonasik wants to merge 2 commits into
mainfrom
partial-cache

Conversation

@samsonasik

Copy link
Copy Markdown
Member

…void partially written cache reads on parallel run
@samsonasik

Copy link
Copy Markdown
Member Author

@dragosprotung could you try manually verify if this works ? Thank you.

@TomasVotruba

Copy link
Copy Markdown
Member

This seems very costly on every cache call.

@TomasVotruba

Copy link
Copy Markdown
Member

@samsonasik

Copy link
Copy Markdown
Member Author

@TomasVotruba we originally follow that, and cause windows issue, see original issue and PR that fix rename issue:

that was resolved by using copy

as back to rename() may cause old bug show again on windows.

@TomasVotruba

Copy link
Copy Markdown
Member

Not sure it was the sole Windows causing issue, that would have been spotted in our CI.
What is the OS causing the issue here?

@samsonasik

samsonasik commented Sep 1, 2026

Copy link
Copy Markdown
Member Author

see still open php issue on rename

we still support php 7.4 on scoped build.

@dragosprotung

Copy link
Copy Markdown
Contributor

@samsonasik using rename works

@samsonasik

samsonasik commented Sep 2, 2026

Copy link
Copy Markdown
Member Author

@TomasVotruba I've updated to php-src include issue:

in the comment and make less diff so easier to review.

Should be ready now 👍

@TomasVotruba

Copy link
Copy Markdown
Member

This still executes a condition on every call. Very costly.

Instead, we should trigger fallback only in case of write failure.
Something like:

$copySuccess = @\copy($tmpPath, $filePath);

if ($copySuccess) {
    return;
}

// try again here

@dragosprotung

Copy link
Copy Markdown
Contributor

Copy does not fail, it's just that in parallel run, the cache file is written by different processes at the same time and it gets corrupted.
copy() truncates the destination and streams into it, so it is not atomic.

@TomasVotruba

Copy link
Copy Markdown
Member

We'll need a failing reproducer in our CI, so we avoid changing this back and forth.

@samsonasik

Copy link
Copy Markdown
Member Author

@TomasVotruba as per @dragosprotung above, this can't one function too early call and fallback later. It not run 2 function, it just verify which OS it run via directory_separator, if it windows, use copy, otherwise, use rename.

The @unlink is original code ensure nothing left behind after copied.

@samsonasik

Copy link
Copy Markdown
Member Author

@TomasVotruba It can't be proven here, since the issue is on php < 8.1 on windows.

@TomasVotruba

TomasVotruba commented Sep 2, 2026

Copy link
Copy Markdown
Member

@samsonasik We can setup any version in CI, even Windows

@samsonasik

samsonasik commented Sep 2, 2026

Copy link
Copy Markdown
Member Author

this repo is on php 8.2, the test require php <8.1, so it needs merge first with e2e on scoped build?

@samsonasik

Copy link
Copy Markdown
Member Author

Even on scoped build, I am not sure if CI can setup UAC protected locations windows setup.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

FileCacheStorage uses copy() instead of rename(), corrupting cache files on parallel cold-cache runs

3 participants