Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Memory leak in file operations in mono 5.10 #7655

Open
andrewaggb opened this issue Mar 16, 2018 · 17 comments

Comments

Projects
None yet
@andrewaggb
Copy link

commented Mar 16, 2018

Steps to Reproduce

  1. Seems to be related to using a lot of FileInfos and/or deleting, copying, appending files. I also access LastWriteTimeUtc alot which I suspect most apps don't.
    Essentially I have approximately 20 ffmpeg processes creating files 5 or so files (per process) every second in a tmpfs mount.

I believe the relevant code is below. Depending on the type of file the file extension in the search may be different but otherwise it's doing the same thing. Finding files older than a certain time and either deleting them outright or combining them into a larger file on another drive and then deleting them.

DirectoryInfo cid = ...
var allfiles = cid.GetFiles("*.0.ts", SearchOption.TopDirectoryOnly).ToArray();
var nowMinus12 = DateTime.UtcNow.AddSeconds(-12);
var files = allfiles.Where(x => x.LastWriteTimeUtc < nowMinus12).OrderBy(x => x.Name).ToArray();
if (!files.Any() || (files[0].LastWriteTimeUtc.AddSeconds(42) >= DateTime.UtcNow)) return;

....
// Note: mfiles is an array of FileInfo

                var finfo = new List<TSFileInfo>();
                using (var fs = File.Open(fname + ".tmp", FileMode.Create))
                {
                    long offset = 0;
                    for (int k = first; k <= last; ++k)
                    {
                        var mfile = mfiles[k];
                        using (var fr = mfile.OpenRead())
                        {
                            CopyStream(fr, fs);
                            finfo.Add(new TSFileInfo() { Offset = offset, tStart = mfile.TSFnCreate() });
                            offset += mfile.Length;
                        }
                        mfile.Delete();
                    }
                    fs.Close();
                }
                File.Move(fname + ".tmp", fname);
                File.WriteAllText(fname + ".json", Newtonsoft.Json.JsonConvert.SerializeObject(finfo));

        public static void CopyStream(Stream input, Stream output)
        {
            byte[] buffer = new byte[65536];
            while (true)
            {
                int read = input.Read(buffer, 0, buffer.Length);
                if (read <= 0)
                    return;
                output.Write(buffer, 0, read);
            }
        }

Some files are copied to another drive and all deleted from tmpfs using the FileInfo Delete method
The actual file writes/copy are done by opening a FileStream on another drive and writing bytes to it.

I could build an app to reproduce the issue if necessary. I've been putting out fires everywhere as it's been crashing my systems (my bad letting yum and apt auto-update systems). My work around for now is to rollback mono on Centos 6 systems and use .net core on Centos 7 or Ubuntu systems.

Current Behavior

As of Mono 5.10 I have an application that leaks memory very fast. It can be using upwards of 80% of system memory according to top after running for a few hours. The same app on the first version of mono 5.8 or older or on .net core holds steady at 1-3% of memory depending on the system (hundreds of installs).

Expected Behavior

Memory usage holds steady instead of constantly increasing over time.

[ ] macOS
[ X] Linux, Centos 6, Centos 7, Ubuntu 16.04
[ ] Windows

Version Used:
Works on mono 4.6.2, 5.2.0, 5.4.0, 5.8.0.108 (unsure about the latest version of 5.8), doesn't work on mono 5.10.0.140 or 5.10.0.160

I'm currently testing to see if mono 5.8.0.127 is affected.

Stacktrace

Please paste the stack trace here if available.
@andrewaggb

This comment has been minimized.

Copy link
Author

commented Mar 17, 2018

This is a simple app to reproduce issue. All this does is create small files, find them, and delete them.

Use a tmpfs mount so it goes faster and you don't wreck your ssd/disk/whatever.
mkdir /tmp/memleaktest
mount -t tmpfs -o defaults,size=64m,noatime tmpfs /tmp/memleaktest
mono FileSystemMemoryLeak.exe /tmp/memleaktest

This zip file has a Visual studio project file, the single source file, and a release build binary from VS 2017. By all means build it yourself. I didn't try building it with mono but I assume that doesn't matter as it only happens on mono 5.10
FileSystemMemoryLeak.zip

Leaks memory pretty fast on mono 5.10.0.160. By the time the loop counter has hit a few thousand it's already doubled it's memory usage. By loop 6000 it's tripled. Very easy to see after a few minutes.

Same test on mono 5.8.0.107 maintains steady memory usage the whole time. As I mentioned earlier in the ticket, earlier versions of mono such as 4.6.2,5.2,5.4 all work with constant memory usage. The same code compiled as a .net core app and run under dotnet 2.0.6 also works fine.

This code is included in the zip file.

using System;
using System.Linq;
using System.IO;
using System.Threading;

namespace FileSystemMemoryLeak
{
    class Program
    {
        static void Main(string[] args)
        {
            if (!args.Any())
            {
                Console.WriteLine(
@"Usage:
    FileSystemMemoryLeak PathToCreateFiles
eg  
Note: Use a tmpfs mount or something similiar as the 'PathToCreateFiles' target as this app will thrash the disk
");
                return;
            }

            if (!Directory.Exists(args[0]))
            {
                Console.WriteLine($"Error: Directory {args[0]} does not exist");
                return;
            }

            var sourceDir = Directory.CreateDirectory(Path.Combine(args[0], "source"));
            var data = new byte[8192];
            long counter = 0;
            while (true)
            {
                try
                {                   
                    //Create nonsense files
                    for (int i=0; i < 1000; ++i)
                    {
                        using (var fs = File.Open(Path.Combine(sourceDir.FullName, i.ToString() + ".tmp"), FileMode.Create))
                        {
                            fs.Write(data, 0, data.Length);
                        }
                    }
                    // find files
                    var files = sourceDir.GetFiles("*.tmp", SearchOption.TopDirectoryOnly);

                    // delete files
                    foreach (var f in files)
                    {
                        f.Delete();
                    }
                    Thread.Sleep(1);
                }
                catch(Exception e)
                {
                    Console.WriteLine(e.ToString());
                }
                if (counter++ % 1000 == 0)
                {
                    Console.WriteLine(DateTime.Now.ToString() + ", Loop Counter: " + counter);
                }
            }
        }
    }
}

@andrewaggb andrewaggb changed the title Apparent Memory leak in file operations in mono 5.10 Memory leak in file operations in mono 5.10 Mar 17, 2018

@marek-safar marek-safar added this to the 2018-02 (5.12.xx) milestone Mar 19, 2018

@marek-safar

This comment has been minimized.

Copy link
Member

commented Mar 19, 2018

I can reproduce the issue and I suspect GetFiles call.

@EgorBo could you try to track it down?

@vargaz

This comment has been minimized.

Copy link
Member

commented Mar 21, 2018

findhandle_close () seems to be missing a
findhandle_unref (findhandle);
call.

@luhenry

This comment has been minimized.

Copy link
Member

commented Mar 21, 2018

@vargaz at https://github.com/mono/mono/blob/master/mono/metadata/w32file-unix.c?utf8=%E2%9C%93#L4706 it should be the following:


static void
finds_remove (gpointer data)
{
	MonoFindHandle* findhandle;

	findhandle = (MonoFindHandle**) data;
	g_assert (findhandle);

	mono_refcount_dec (findhandle);
}

...

finds = g_hash_table_new_full (g_direct_hash, g_direct_equal, NULL, finds_remove);
@HeikoWeiss

This comment has been minimized.

Copy link

commented Mar 22, 2018

seems to be fixed with my test in
#7734

Please check it.
Thank You.

@pr0vieh

This comment has been minimized.

Copy link

commented Mar 23, 2018

@HeikoWeiss i think it's a good idea to open a Pull Req for your fix then you get feedback if your Fix is the right solution of this Bug

@marek-safar

This comment has been minimized.

Copy link
Member

commented Apr 5, 2018

@luhenry could you turn that into PR?

luhenry added a commit to luhenry/mono that referenced this issue Apr 6, 2018

@luhenry luhenry closed this in #8090 Apr 7, 2018

luhenry added a commit that referenced this issue Apr 7, 2018

monojenkins added a commit to monojenkins/mono that referenced this issue Apr 9, 2018

monojenkins added a commit to monojenkins/mono that referenced this issue Apr 9, 2018

luhenry added a commit that referenced this issue Apr 9, 2018

akoeplinger added a commit that referenced this issue Apr 10, 2018

jonpryor added a commit to xamarin/xamarin-android that referenced this issue Apr 25, 2018

Bump to mono/2018-02/0c5a524e (#1289)
Bumps to Java.Interop/master/0afb2b0f
Bumps to llvm/master/a9cfb50e.

Fixes: https://bugzilla.xamarin.com/show_bug.cgi?id=11771
Fixes: https://bugzilla.xamarin.com/show_bug.cgi?id=15051
Fixes: https://bugzilla.xamarin.com/show_bug.cgi?id=19436
Fixes: https://bugzilla.xamarin.com/show_bug.cgi?id=45901
Fixes: https://bugzilla.xamarin.com/show_bug.cgi?id=56071
Fixes: https://bugzilla.xamarin.com/show_bug.cgi?id=58413
Fixes: https://bugzilla.xamarin.com/show_bug.cgi?id=58413
Fixes: https://bugzilla.xamarin.com/show_bug.cgi?id=58413
Fixes: https://bugzilla.xamarin.com/show_bug.cgi?id=59184
fixes: https://bugzilla.xamarin.com/show_bug.cgi?id=60065
Fixes: https://bugzilla.xamarin.com/show_bug.cgi?id=60225
Fixes: https://bugzilla.xamarin.com/show_bug.cgi?id=60298
Fixes: https://bugzilla.xamarin.com/show_bug.cgi?id=60359
Fixes: https://bugzilla.xamarin.com/show_bug.cgi?id=60568
Fixes: https://bugzilla.xamarin.com/show_bug.cgi?id=60756
Fixes: https://bugzilla.xamarin.com/show_bug.cgi?id=60848
Fixes: https://bugzilla.xamarin.com/show_bug.cgi?id=60862
Fixes: https://bugzilla.xamarin.com/show_bug.cgi?id=60900
Fixes: https://bugzilla.xamarin.com/show_bug.cgi?id=60904
Fixes: https://bugzilla.xamarin.com/show_bug.cgi?id=60986
Fixes: https://github.com/mono/mono/issues/59400
Fixes: mono/mono#6169
Fixes: mono/mono#6187
Fixes: mono/mono#6192
Fixes: mono/mono#6255
Fixes: mono/mono#6264
Fixes: mono/mono#6266
Fixes: mono/mono#6281
Fixes: mono/mono#6283
Fixes: mono/mono#6320
Fixes: mono/mono#6339
Fixes: mono/mono#6343
Fixes: mono/mono#6349
Fixes: mono/mono#6379
Fixes: mono/mono#6383
Fixes: mono/mono#6401.
Fixes: mono/mono#6411
Fixes: mono/mono#6414
Fixes: mono/mono#6490
Fixes: mono/mono#6721
Fixes: mono/mono#6767
Fixes: mono/mono#6777
Fixes: mono/mono#6848
Fixes: mono/mono#6940
Fixes: mono/mono#6948
Fixes: mono/mono#6998
Fixes: mono/mono#7016
Fixes: mono/mono#7085
Fixes: mono/mono#7086
Fixes: mono/mono#7095
Fixes: mono/mono#7137
Fixes: mono/mono#7184
Fixes: mono/mono#7240
Fixes: mono/mono#7262
Fixes: mono/mono#7289
Fixes: mono/mono#7338
Fixes: mono/mono#7356
Fixes: mono/mono#7364
Fixes: mono/mono#7378
Fixes: mono/mono#7389
Fixes: mono/mono#7460
Fixes: mono/mono#7535
Fixes: mono/mono#7536
Fixes: mono/mono#7610
Fixes: mono/mono#7624
Fixes: mono/mono#7637
Fixes: mono/mono#7655
Fixes: mono/mono#7657
Fixes: mono/mono#7685
Fixes: mono/mono#7786
Fixes: mono/mono#7792
Fixes: mono/mono#7822
Fixes: mono/mono#7860
Fixes: mono/mono#8089
Fixes: mono/mono#8267
Fixes: mono/mono#8409
Fixes: xamarin/maccore#628
Fixes: xamarin/maccore#629
Fixes: xamarin/maccore#673
Fixes: xamarin/maccore#673
Fixes: #1561

jonpryor added a commit to xamarin/xamarin-android that referenced this issue Aug 8, 2018

Bump to mono/mono:2018-04@f3a2216b (#1503)
Fixes: #1130
Fixes: #1561 (comment)
Fixes: #1845
Fixes: #1951

Context: https://bugzilla.xamarin.com/show_bug.cgi?id=10087
Context: https://bugzilla.xamarin.com/show_bug.cgi?id=11771
Context: https://bugzilla.xamarin.com/show_bug.cgi?id=12850
Context: https://bugzilla.xamarin.com/show_bug.cgi?id=18941
Context: https://bugzilla.xamarin.com/show_bug.cgi?id=19436
Context: https://bugzilla.xamarin.com/show_bug.cgi?id=25444
Context: https://bugzilla.xamarin.com/show_bug.cgi?id=33208
Context: https://bugzilla.xamarin.com/show_bug.cgi?id=58413
Context: https://bugzilla.xamarin.com/show_bug.cgi?id=59184
Context: https://bugzilla.xamarin.com/show_bug.cgi?id=59400
Context: https://bugzilla.xamarin.com/show_bug.cgi?id=59779
Context: https://bugzilla.xamarin.com/show_bug.cgi?id=60065
Context: https://bugzilla.xamarin.com/show_bug.cgi?id=60843
Context: mono/mono#6174
Context: mono/mono#6178
Context: mono/mono#6180
Context: mono/mono#6181
Context: mono/mono#6186
Context: mono/mono#6187
Context: mono/mono#6211
Context: mono/mono#6266
Context: mono/mono#6579
Context: mono/mono#6666
Context: mono/mono#6752
Context: mono/mono#6801
Context: mono/mono#6812
Context: mono/mono#6848
Context: mono/mono#6940
Context: mono/mono#6948
Context: mono/mono#6998
Context: mono/mono#6999
Context: mono/mono#7016
Context: mono/mono#7085
Context: mono/mono#7086
Context: mono/mono#7095
Context: mono/mono#7134
Context: mono/mono#7137
Context: mono/mono#7145
Context: mono/mono#7184
Context: mono/mono#7240
Context: mono/mono#7262
Context: mono/mono#7289
Context: mono/mono#7338
Context: mono/mono#7356
Context: mono/mono#7364
Context: mono/mono#7378
Context: mono/mono#7389
Context: mono/mono#7449
Context: mono/mono#7460
Context: mono/mono#7535
Context: mono/mono#7536
Context: mono/mono#7537
Context: mono/mono#7565
Context: mono/mono#7588
Context: mono/mono#7596
Context: mono/mono#7610
Context: mono/mono#7613
Context: mono/mono#7620
Context: mono/mono#7624
Context: mono/mono#7637
Context: mono/mono#7655
Context: mono/mono#7657
Context: mono/mono#7661
Context: mono/mono#7685
Context: mono/mono#7696
Context: mono/mono#7729
Context: mono/mono#7786
Context: mono/mono#7792
Context: mono/mono#7805
Context: mono/mono#7822
Context: mono/mono#7828
Context: mono/mono#7860
Context: mono/mono#7864
Context: mono/mono#7903
Context: mono/mono#7920
Context: mono/mono#8089
Context: mono/mono#8143
Context: mono/mono#8267
Context: mono/mono#8311
Context: mono/mono#8340
Context: mono/mono#8409
Context: mono/mono#8417
Context: mono/mono#8430
Context: mono/mono#8698
Context: mono/mono#8701
Context: mono/mono#8712
Context: mono/mono#8721
Context: mono/mono#8726
Context: mono/mono#8866
Context: mono/mono#9023
Context: mono/mono#9031
Context: mono/mono#9033
Context: mono/mono#9044
Context: mono/mono#9179
Context: mono/mono#9318
Context: mono/mono#9318
Context: xamarin/maccore#628
Context: xamarin/maccore#629
Context: xamarin/maccore#673

jonpryor added a commit to xamarin/xamarin-android that referenced this issue Aug 13, 2018

Bump to mono/mono:2018-04@f3a2216b (#1503)
Fixes: #1130
Fixes: #1561 (comment)
Fixes: #1845
Fixes: #1951

Context: https://bugzilla.xamarin.com/show_bug.cgi?id=10087
Context: https://bugzilla.xamarin.com/show_bug.cgi?id=11771
Context: https://bugzilla.xamarin.com/show_bug.cgi?id=12850
Context: https://bugzilla.xamarin.com/show_bug.cgi?id=18941
Context: https://bugzilla.xamarin.com/show_bug.cgi?id=19436
Context: https://bugzilla.xamarin.com/show_bug.cgi?id=25444
Context: https://bugzilla.xamarin.com/show_bug.cgi?id=33208
Context: https://bugzilla.xamarin.com/show_bug.cgi?id=58413
Context: https://bugzilla.xamarin.com/show_bug.cgi?id=59184
Context: https://bugzilla.xamarin.com/show_bug.cgi?id=59400
Context: https://bugzilla.xamarin.com/show_bug.cgi?id=59779
Context: https://bugzilla.xamarin.com/show_bug.cgi?id=60065
Context: https://bugzilla.xamarin.com/show_bug.cgi?id=60843
Context: mono/mono#6174
Context: mono/mono#6178
Context: mono/mono#6180
Context: mono/mono#6181
Context: mono/mono#6186
Context: mono/mono#6187
Context: mono/mono#6211
Context: mono/mono#6266
Context: mono/mono#6579
Context: mono/mono#6666
Context: mono/mono#6752
Context: mono/mono#6801
Context: mono/mono#6812
Context: mono/mono#6848
Context: mono/mono#6940
Context: mono/mono#6948
Context: mono/mono#6998
Context: mono/mono#6999
Context: mono/mono#7016
Context: mono/mono#7085
Context: mono/mono#7086
Context: mono/mono#7095
Context: mono/mono#7134
Context: mono/mono#7137
Context: mono/mono#7145
Context: mono/mono#7184
Context: mono/mono#7240
Context: mono/mono#7262
Context: mono/mono#7289
Context: mono/mono#7338
Context: mono/mono#7356
Context: mono/mono#7364
Context: mono/mono#7378
Context: mono/mono#7389
Context: mono/mono#7449
Context: mono/mono#7460
Context: mono/mono#7535
Context: mono/mono#7536
Context: mono/mono#7537
Context: mono/mono#7565
Context: mono/mono#7588
Context: mono/mono#7596
Context: mono/mono#7610
Context: mono/mono#7613
Context: mono/mono#7620
Context: mono/mono#7624
Context: mono/mono#7637
Context: mono/mono#7655
Context: mono/mono#7657
Context: mono/mono#7661
Context: mono/mono#7685
Context: mono/mono#7696
Context: mono/mono#7729
Context: mono/mono#7786
Context: mono/mono#7792
Context: mono/mono#7805
Context: mono/mono#7822
Context: mono/mono#7828
Context: mono/mono#7860
Context: mono/mono#7864
Context: mono/mono#7903
Context: mono/mono#7920
Context: mono/mono#8089
Context: mono/mono#8143
Context: mono/mono#8267
Context: mono/mono#8311
Context: mono/mono#8340
Context: mono/mono#8409
Context: mono/mono#8417
Context: mono/mono#8430
Context: mono/mono#8698
Context: mono/mono#8701
Context: mono/mono#8712
Context: mono/mono#8721
Context: mono/mono#8726
Context: mono/mono#8866
Context: mono/mono#9023
Context: mono/mono#9031
Context: mono/mono#9033
Context: mono/mono#9044
Context: mono/mono#9179
Context: mono/mono#9318
Context: mono/mono#9318
Context: xamarin/maccore#628
Context: xamarin/maccore#629
Context: xamarin/maccore#673

jonpryor added a commit to xamarin/xamarin-android that referenced this issue Oct 9, 2018

Bump to mono/mono:2018-06@ab3c897d (#1836)
Bumps to mono/llvm:release_60@117a508c
Bumps to xamarin/xamarin-android-api-compatibility:master@7ccb4802

	$ git diff --shortstat e1af6ea..ab3c897d       # mono
        1443 files changed, 66049 insertions(+), 45745 deletions(-)
	$ git diff --shortstat bdb3a116..117a508c      # llvm
	 26794 files changed, 4110589 insertions(+), 754376 deletions(-)
	$ git diff --shortstat c550d1bd..7ccb4802      # xamarin-android-api-compatibility
	 2 files changed, 16260 insertions(+), 12347 deletions(-)

Incomplete summary of easily `grep`able fixes:

Fixes: https://bugzilla.xamarin.com/show_bug.cgi?id=11199
Fixes: https://bugzilla.xamarin.com/show_bug.cgi?id=19436
Fixes: https://bugzilla.xamarin.com/show_bug.cgi?id=23668
Fixes: https://bugzilla.xamarin.com/show_bug.cgi?id=26983
Fixes: https://bugzilla.xamarin.com/show_bug.cgi?id=33728
Fixes: https://bugzilla.xamarin.com/show_bug.cgi?id=46917
fixes: https://bugzilla.xamarin.com/show_bug.cgi?id=60065
Fixes: mono/mono#6173
Fixes: mono/mono#6466
Fixes: mono/mono#6647
Fixes: mono/mono#6834
Fixes: mono/mono#7058
Fixes: mono/mono#7137
Fixes: mono/mono#7260
Fixes: mono/mono#7305
Fixes: mono/mono#7402
Fixes: mono/mono#7525
Fixes: mono/mono#7610
Fixes: mono/mono#7649
Fixes: mono/mono#7655
Fixes: mono/mono#7683
Fixes: mono/mono#7685
Fixes: mono/mono#7716
Fixes: mono/mono#7731
Fixes: mono/mono#7785
Fixes: mono/mono#7828
Fixes: mono/mono#7944
Fixes: mono/mono#7947
Fixes: mono/mono#8036
Fixes: mono/mono#8074
Fixes: mono/mono#8089
Fixes: mono/mono#8112
Fixes: mono/mono#8122
Fixes: mono/mono#8143
Fixes: mono/mono#8149
Fixes: mono/mono#8152
Fixes: mono/mono#8175
Fixes: mono/mono#8177
Fixes: mono/mono#8250
Fixes: mono/mono#8267
Fixes: mono/mono#8273
Fixes: mono/mono#8282
Fixes: mono/mono#8310
Fixes: mono/mono#8311
Fixes: mono/mono#8329
Fixes: mono/mono#8340
Fixes: mono/mono#8372
Fixes: mono/mono#8407
Fixes: mono/mono#8409
Fixes: mono/mono#8422
Fixes: mono/mono#8430
Fixes: mono/mono#8439
fixes: mono/mono#8447
Fixes: mono/mono#8469
Fixes: mono/mono#8504
Fixes: mono/mono#8575
Fixes: mono/mono#8597
Fixes: mono/mono#8623
Fixes: mono/mono#8627
Fixes: mono/mono#8698
Fixes: mono/mono#8701
Fixes: mono/mono#8712
Fixes: mono/mono#8721
Fixes: mono/mono#8726
Fixes: mono/mono#8759
Fixes: mono/mono#8787
Fixes: mono/mono#8820
Fixes: mono/mono#8848
Fixes: mono/mono#8866
Fixes: mono/mono#8897
Fixes: mono/mono#8915
Fixes: mono/mono#8970
Fixes: mono/mono#8979
Fixes: mono/mono#9023
Fixes: mono/mono#9031
Fixes: mono/mono#9033
Fixes: mono/mono#9179
Fixes: mono/mono#9234
Fixes: mono/mono#9262
Fixes: mono/mono#9277
Fixes: mono/mono#9318
Fixes: mono/mono#9542
Fixes: mono/mono#9753
Fixes: mono/mono#9839
Fixes: mono/mono#9869
Fixes: mono/mono#9870
Fixes: mono/mono#9943
Fixes: mono/mono#9996
Fixes: mono/mono#10000
Fixes: mono/mono#10303
Fixes: mono/mono#10447
Fixes: mono/mono#10483
Fixes: mono/mono#10488
Fixes: xamarin/maccore#628
Fixes: xamarin/maccore#673
Fixes: #1561 (comment)
Fixes: #1845
Fixes: xamarin/xamarin-macios#4347
Fixes: xamarin/xamarin-macios#4617
Fixes: xamarin/xamarin-macios#4618
@schtibb

This comment has been minimized.

Copy link

commented Oct 15, 2018

We have to use mono 5.12 in our production environment which is affected by this memory leak. If we use the nuget Mono Posix package and Mono.Unix.UnixDirectoryInfo.GetFileSystemEntries instead of System.IO.DirectoryInfo.GetFiles will that help? We are running mono on Linux

@adsbxchange

This comment has been minimized.

Copy link

commented Mar 5, 2019

Still not fixed. Using 5.8.0.127 in production, any later mono leaks memory and crashes out application.

@avkhitry

This comment has been minimized.

Copy link

commented Mar 10, 2019

I have similar problem with memory leak during file operations, any chance it will be fixed?

@adsbxchange

This comment has been minimized.

Copy link

commented Apr 21, 2019

This needs to be reopened and addressed.

@lewurm lewurm reopened this Apr 23, 2019

@lewurm

This comment has been minimized.

Copy link
Member

commented Apr 23, 2019

@adsbxchange the fix made it into 5.12 and later. can you try a newer mono?

@adsbxchange

This comment has been minimized.

Copy link

commented Apr 23, 2019

@adsbxchange

This comment has been minimized.

Copy link

commented Apr 25, 2019

vradarserver/vrs#11

VRS dev is salty about it. Not sure where the issue is. But being that is been an issue through multiple releases, there should be some fix.

@lewurm

This comment has been minimized.

Copy link
Member

commented May 8, 2019

@adsbxchange in order to track this down we need a reproducer. The repro from #7655 (comment) works fine for me on 6.0.

@adsbxchange

This comment has been minimized.

Copy link

commented May 8, 2019

@adsbxchange in order to track this down we need a reproducer. The repro from #7655 (comment) works fine for me on 6.0.

then it might not be the same issue or it's only related ... VRS (virtualradarserver.co.uk) is built in .NET and opens a lot of file descriptors and only leaks in mono after 5.8.0.127

Easily reproducible when using Mono.

Hopefully that helps ...

@emmanuelmathot

This comment has been minimized.

Copy link

commented Jul 11, 2019

using valgrind to perform deep memory leak check, I can confirm that huge memory leaks happen with mono > 5.8.0.108

When method uses a initialized FileInfo class with mono 5.20.1.19:

==28074== LEAK SUMMARY:
==28074==    definitely lost: 6,533,670 bytes in 62,818 blocks
==28074==    indirectly lost: 35,549 bytes in 364 blocks
==28074==      possibly lost: 42,287 bytes in 116 blocks
==28074==    still reachable: 68,151,840 bytes in 442,799 blocks
==28074==                       of which reachable via heuristic:
==28074==                         stdstring          : 19,308 bytes in 576 blocks
==28074==         suppressed: 0 bytes in 0 blocks
==28074== Reachable blocks (those to which a pointer was found) are not shown.
==28074== To see them, rerun with: --leak-check=full --show-leak-kinds=all

same program with mono 5.8.0.108:

==15143== LEAK SUMMARY:
==15143==    definitely lost: 95 bytes in 2 blocks
==15143==    indirectly lost: 0 bytes in 0 blocks
==15143==      possibly lost: 13,952 bytes in 22 blocks
==15143==    still reachable: 61,483,893 bytes in 294,702 blocks
==15143==                       of which reachable via heuristic:
==15143==                         stdstring          : 19,694 bytes in 581 blocks
==15143==         suppressed: 0 bytes in 0 blocks
==15143== Rerun with --leak-check=full to see details of leaked memory
==15143== 
==15143== For counts of detected and suppressed errors, rerun with: -v
==15143== ERROR SUMMARY: 3920 errors from 53 contexts (suppressed: 0 from 0)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.