Join GitHub today
GitHub is home to over 20 million developers working together to host and review code, manage projects, and build software together.
Large number of async file writes cause data corruption #808
I've been trying to generate a large file, writing one line at a time, and I ran into a serious problem. Here's a test case (original CoffeeScript gisted here:
This will simply write ten million random numbers to a file, separated by newlines. Try running it; it'll take 15+ minutes to complete. Then go to the command line and do
This will match several lines starting with more than one decimal digit, which should be impossible (in fact, the code that wrote those lines would have written to the console to alert us, because such a line doesn't match
...and so on for about 50 lines.
I've done repeated tests, and this never happens when I do a million writes. It's that jump to 10,000,000 that does it (even though the bad writes can, as you see above, occur well before the millionth one)!
For the record, I'm running Node 0.4.2 under Mac OS 10.6.6, with an SSD. I haven't run this test on any other systems.