WindowsMultiProcessFileAppender - FileSystemRights.AppendData

[snakefoot]

Merged #1474 with master, and changed from native windows file handle to managed FileStream-object

Using FileSystemRights.AppendData (FILE_APPEND_DATA / O_APPEND) allows multiple processes to write to the same local file without needing a global mutex object. This improves file-write performance with a factor 2.

There are different rumors on the Internet whether FILE_APPEND_DATA works correctly when the number of bytes written in a single operation is larger than 1KByte or 4KByte. The file-size increase that allocates the needed room in the file is atomic. But writing the bytes to the allocated room is probably done in sector size depending on version of operating system. This means if having very aggressive tail application that performs direct file reads, then it can momentarily read the allocated room without data has actually been written.

---

This change is

[snakefoot]

I have tested the updated solution with the test-project from #1474 made by @AndreGleichner, and it completes and performs successful verification.

[codecov-io]

Current coverage is 81% (diff: 82%)

Merging #1756 into master will increase coverage by <1%

@@             master      #1756   diff @@
==========================================
  Files           276        277     +1   
  Lines         17523      17626   +103   
  Methods        2756       2767    +11   
  Messages          0          0          
  Branches       1975       1989    +14   
==========================================
+ Hits          14089      14226   +137   
+ Misses         2988       2947    -41   
- Partials        446        453     +7   

Powered by Codecov. Last update cd62cd0...80e40da

[304NotModified]

There are different rumors on the Internet whether FILE_APPEND_DAT

So it's untrue? Or unsure?

[304NotModified]

Thanks for the changes! Will try to review it soon and release in into 4.4 (rc)

[snakefoot]

Have noticed that the unit test time has increased. Most likely because of the added combinations in FileTargetTests.cs (Linq-booleanValues).

Not sure if this is a good direction, as every new parameter will double the time.

[304NotModified]

Good to know. It was ~11min and now ~19.

In my experience 11 was already to long. When working on several items, I have to wait for hours.

Everything is tested with net35+net40+net45. Maybe we should test some tests only with net45

[snakefoot]

Added a filter so it only exercises the major FileAppenders-combinations. Though it just removed 1-2 minutes.

Travis Test Count = 1770 (Before=2190), but still an increase from current master=1554
AppVeyor Test Count = 1915 (Before=2355), but still an increase from current master=1677

It seems the time is spend by the code-coverage, as the other builds completes quickly, so not sure if the skipping the FileTargetTest for the other builds will help that much.

[304NotModified]

Will try to test this next week

[304NotModified]

Reviewed 1 of 20 files at r1, 23 of 24 files at r2.
Review status: all files reviewed at latest revision, 1 unresolved discussion.

---

src/NLog/Internal/FileCharacteristicsHelper.cs, line 68 at r2 (raw file):

        /// <param name="fileStream">The file stream.</param>
        /// <returns>The file characteristics, if the file information was retrieved successfully, otherwise null.</returns>
        public abstract FileCharacteristics GetFileCharacteristics(string fileName, FileStream fileStream);

Is this not infecting performance?

---

Comments from Reviewable

[304NotModified]

Need docs for

• ForceManaged ,
• ForceMutexConcurrentWrites

[snakefoot]

src/NLog/Internal/FileCharacteristicsHelper.cs, line 68 at r2 (raw file):

Previously, 304NotModified (Julian Verdurmen) wrote…

> Is this not infecting performance?

Changing the parameter from IntPtr to FileStream-reference ?

---

Comments from Reviewable

[304NotModified]

Review status: all files reviewed at latest revision, 1 unresolved discussion.

---

src/NLog/Internal/FileCharacteristicsHelper.cs, line 68 at r2 (raw file):

Previously, snakefoot (Rolf Kristensen) wrote…

> Changing the parameter from IntPtr to FileStream-reference ?

Well using FileStream insteaf of IntPtr

---

Comments from Reviewable

[snakefoot]

src/NLog/Internal/FileCharacteristicsHelper.cs, line 68 at r2 (raw file):

Previously, 304NotModified (Julian Verdurmen) wrote…

> Well using FileStream insteaf of IntPtr

I'm only using the FileStream-parameter to carry the IntPtr (If needed). If using Win32FileCharacteristicsHelper then it will use fileStream.SafeFileHandle.DangerousGetHandle()

---

Comments from Reviewable

[304NotModified]

Review status: all files reviewed at latest revision, 1 unresolved discussion.

---

src/NLog/Internal/FileCharacteristicsHelper.cs, line 68 at r2 (raw file):

Previously, snakefoot (Rolf Kristensen) wrote…

> I'm only using the FileStream-parameter to carry the IntPtr (If needed). If using Win32FileCharacteristicsHelper then it will use fileStream.SafeFileHandle.DangerousGetHandle()

OK thanks! :)

---

Comments from Reviewable

[304NotModified]

from tests give me ~10% performance gain, also with 1 thread.

tested with

NLogPerformance\bin\Release\NLogPerformance.exe 3000000 1  8  1 false >> performance-log.log
NLogPerformance\bin\Release\NLogPerformance.exe 3000000 1  16 1 false >> performance-log.log
NLogPerformance\bin\Release\NLogPerformance.exe 3000000 1  32 1 false >> performance-log.log
NLogPerformance\bin\Release\NLogPerformance.exe 3000000 2  16 1 false >> performance-log.log
NLogPerformance\bin\Release\NLogPerformance.exe 3000000 4  16 1 false >> performance-log.log
NLogPerformance\bin\Release\NLogPerformance.exe 3000000 8  16 1 false >> performance-log.log
NLogPerformance\bin\Release\NLogPerformance.exe 3000000 16 16 1 false >> performance-log.log
NLogPerformance\bin\Release\NLogPerformance.exe 3000000 60 16 1 false >> performance-log.log

[304NotModified]

Will be in news post