Infrequent caching error for modSettings

[sbulen]

Description

Error message:
PHP Parse error: syntax error, unexpected ''a:379:{s:10:"smfVersion";s:6:' (T_ENCAPSED_AND_WHITESPACE) in /home/blahblah/public_html/cache/data_e5def45bc64284f0d829317a2d0686be-SMF-modSettings.php on line 1odSettings.php on line 1
This error has been reported multiple times for both 2.0 & 2.1. For some users (such as myself) the error message is fleeting, and there is no apparent impact on users. For some forums, however, users are locked out of the forum until cache is cleared.

My system uses simple SMF file caching, level 1, with no OS or host-level caching.

I suspect that an incomplete read is occurring while the file is being written. Note the read is done via an "include" in 2.0 and 2.1 - same technique. I suspect that for some users, the incomplete file that was read gets cached at the OS or host level, causing users to get locked out.

Steps to reproduce

1. This is intermittent & very infrequent... Difficult to reproduce.

Environment (complete as necessary)

• Version/Git revision: various
• Database Type: various
• Database Version: various
• PHP Version: various

Additional information/references

I also logged this issue in the 2.0 issue log.

Sample threads:
https://www.simplemachines.org/community/index.php?topic=569135.0
https://www.simplemachines.org/community/index.php?topic=563664.0
https://www.simplemachines.org/community/index.php?topic=567903.0
https://www.simplemachines.org/community/index.php?topic=567931.0

[albertlast]

is only the smf file cache affected by this issue?

[sbulen]

I believe that is the case. It uses an "include" to read the file; I doubt there is any kind of locking or waiting there. My current theory is that the file is written OK, it is that the read may get a partial file.

[VBGAMER45]

Why do you think it only happens with the SMF-modSettings.php cache file?

[sbulen]

Good question! Size (multi blocks to read)? Frequency of update? Dunno...

[albertlast]

i guess too,
that the nature of modsetting is that it get update very often and
the smf file cache is not handling multiple access/writing well.

[sbulen]

Another occurrence:
https://www.simplemachines.org/community/index.php?topic=570852.0

[jdarwood007]

I wonder if we can test filesize in addition to the returned result from file_put_contents.

[sbulen]

This user caught an example & shared it in the thread. The file is simply truncated. It's not a bad data element (meaning, it's not that there is a bad data value there).

It's got something to do with i/o - the entire file was not written/read.

Here's the end of the file (I added newlines to make this readable):

Again, I think we need to use a mechanism other than include/require to read these. I believe that include is intended for infrequent updates, quick reads (source code). The locking mechanism is unknown; locking is probably not even used.

[jdarwood007]

Include was supposed to be used to allow us to do a non fatal error and load it multiple times if needed.

The code does a read of the bytes returned by file_put_contents and compares the bytes written versus the string length which should equal. So the only other thing I could think of is check filesize as well which should query the disks for the file size, rather than depending on what the file system said it wrote (even if it failed to write).

[jdarwood007]

We should test if try/catch would be a safer as well.

[sbulen]

Include was supposed to be used to allow us to do a non fatal error and load it multiple times if needed.

The code does a read of the bytes returned by file_put_contents and compares the bytes written versus the string length which should equal. So the only other thing I could think of is check filesize as well which should query the disks for the file size, rather than depending on what the file system said it wrote (even if it failed to write).

I don't think that is going to work. What we've seen here is that we can, under some circumstances, actually get a parse error. That's because include/require executes the code before you can even do any checks.

Maybe if we suppress parse errors we can stick with include/require? But at that point, we're trying too hard to make include/require work. The bottom line is that this isn't what includes are designed to do...

Normal file locking is intended to deal with this. And it can also trap errors, & also provide multiple reads if needed.

[Arantor]

Or, don’t cache modSettings. If you must cache, cache it only to backends that have actually atomic consistency at the write level (I.e. everything except file cache)

[sbulen]

...or stop putting volatile content in it.

[Arantor]

Well, giving a TTL of 90 seconds with a variety of invalidation throughout the system, that’s not exactly volatile by definition, because it’s not the content of modSettings that’s inherently volatile. It’s the short TTL and implied use of it as a cache invalidator for other things.

Seriously, if this were in Redis, your problem goes away, not because it isn’t volatile but because you have consistency guarantees; your issue is a race condition and file systems (especially NFS) not giving you consistency guarantees. Then again, I’ve seen file_exists return false for files I can see on the file system, so... if you rely on properties of implementation of the file system for data consistency, it will end badly.

Doing an opendir call after the write but before a check against the file size check may insulate you some. Wrapping it in a try/catch guarantees on PHP 7+ that it won’t kill the page since ParseError is catchable in 7+.

Note that for people who get this, it tends to recur, so it’s environmental.