Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FileStream] Flush(flushToDisk: true) should always flush to disk #77384

Merged
merged 3 commits into from
Oct 26, 2022

Conversation

adamsitnik
Copy link
Member

@adamsitnik adamsitnik commented Oct 24, 2022

fixes #77373

5.0 implementation for reference:

internal override void Flush(bool flushToDisk)
{
FlushInternalBuffer();
if (flushToDisk && CanWrite)
{
FileStreamHelpers.FlushToDisk(_fileHandle);
}
}

I've tried really hard to write a reliable test for this bug, but I've failed to do that in reasonable period of time.

Based on the docs:

File system metadata is always cached. Therefore, to store any metadata changes to disk, the file must either be flushed or be opened with FILE_FLAG_WRITE_THROUGH.

I've tried to find some way of writing and reading the metadata and using Flush(true) to flush the metadata, but I was unable to find a simple and reliable solution for doing that (the metadata that we expose is refreshed without the need to flush to disk, other metdata is hard to get and either requires some dirty COM code or 3rd party or .NET Framework-only dependency to do it).

@adamsitnik adamsitnik added this to the 8.0.0 milestone Oct 24, 2022
@ghost ghost assigned adamsitnik Oct 24, 2022
@ghost
Copy link

ghost commented Oct 24, 2022

Tagging subscribers to this area: @dotnet/area-system-io
See info in area-owners.md if you want to be subscribed.

Issue Details

fixes #77373

6.0 implementation for reference:

internal override void Flush(bool flushToDisk)
{
FlushInternalBuffer();
if (flushToDisk && CanWrite)
{
FileStreamHelpers.FlushToDisk(_fileHandle);
}
}

I've tried really hard to write a reliable test for this bug, but I've failed to do that in reasonable period of time.

Based on the docs:

File system metadata is always cached. Therefore, to store any metadata changes to disk, the file must either be flushed or be opened with FILE_FLAG_WRITE_THROUGH.

I've tried to find some way of writing and reading the metadata and using Flush(true) to flush the metadata, but I was unable to find a simple and reliable solution for doing that (the metadata that we expose is refreshed without the need to flush to disk, other metdata is hard to get and either requires some dirty COM code or 3rd party or .NET Framework-only dependency to do it).

Author: adamsitnik
Assignees: -
Labels:

area-System.IO

Milestone: 8.0.0

// If the underlying strategy is not seekable AND we have something in the read buffer, then FlushRead would throw.
// We can either throw away the buffer resulting in data loss (!) or ignore the Flush.
// We cannot throw because it would be a breaking change. We opt into ignoring the Flush in that situation.
return;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why shouldn't / can't we still FSync / FlushFileBuffers in this case?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why shouldn't / can't we still FSync / FlushFileBuffers in this case?

I would expect it to throw for pipes. I am going to check it and get back to you.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@stephentoub it turned out to be possible, moreover we even already have tests for that:

[Theory]
[InlineData(null)]
[InlineData(false)]
[InlineData(true)]
[SkipOnPlatform(TestPlatforms.Browser, "IO.Pipes not supported")]
public void FlushCanBeUsedOnPipes_Success(bool? flushToDisk)
{
using (var pipeStream = new AnonymousPipeServerStream(PipeDirection.In))
using (var clientHandle = pipeStream.ClientSafePipeHandle)
{
SafeFileHandle handle = new SafeFileHandle((IntPtr)int.Parse(pipeStream.GetClientHandleAsString()), false);
using (FileStream fs = new FileStream(handle, FileAccess.Write, 1, false))
{
Flush(fs, flushToDisk);
}
}
}

I've updated the code.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for checking.

@adamsitnik adamsitnik merged commit 3d29176 into dotnet:main Oct 26, 2022
@ghost ghost locked as resolved and limited conversation to collaborators Nov 26, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Buffered file stream doesn't flush data right
3 participants