[Question] Memory leak in Join method

[MakarovSoftware]

Just for test take first test method (Join()), and run it several times (10 000, 100 000, etc.), like this. And every next iteration will take a little bit longer to pass through + we'll get memory leaks. For example, first time it will take about 00:00:00.003, And on 12000 iteration it will take 00:00:00.101 already. On iteration 27400 it will take 00:00:00.214 and so on..

But it's on a small data. In a real project (in my case), for example every next Join gets about 5sec longer:
1 iteration = 3.7 sec
2 iteration = 8 sec
3 iteration = 13 sec
4 iteration = 19 sec.

And now it's a critical bug for me. I think that there is some problems with static cache.

[StefH]

Maybe related to #84 ?

[MakarovSoftware]

Yep, I thought about it, but I'm not sure. Anyway, please, fix this ASAP :)) Thank you!

[StefH]

Please test latest NuGet version1.0.7.3

[MakarovSoftware]

Thank you, I've tested it now. Now it soo much better, but still leaks:

1 iteration = 0.95 sec (fantastic!)
2 iteration = 1.4 sec
3 iteration = 1.9 sec
4 iteration = 2.4 sec
5 iteration = 2.98 sec
36 iteration = 21 sec

Well, here in every iteration I work wit exactly the same classes, that connected together through lots of JOINs, it's all the same, but I change just a condition of WHERE statement from iteration to iteration. Hope, this will help.

[StefH]

Can you add this example project on github? That makes it easier for me to analyse and debug.

[MakarovSoftware]

The problem is that it's not an example project, I tell you timers from real app and it uses big data with lot's of JOINS, and of course, I can't upload it. But ok, I'll try to make some demo test for this case

[MakarovSoftware]

Hi! I've made some demo, try to download source from my repo on github, dunno if I done everything right. There is a simple console app, and you could see the growth of time and memory :) Looking forward to your answer!

[StefH]

Are you sure you have tested the latest version : 1.0.7.3 ?

Because when I run your example, I see this:

press key to run
 0 - 251
10 - 6
20 - 8
30 - 4
40 - 3
50 - 9
60 - 2
70 - 7
80 - 2
90 - 4
100 - 2
110 - 12
120 - 2
130 - 2
140 - 3

[MakarovSoftware]

Ok... very strange, on 1.0.7.3 it really works. I'll check this on a real project today and let you know how it goes

[MakarovSoftware]

Ok, finally I got it :) Try out my last commit, but not sure that you'll think it's a bug :) The problem was that I've generated random names every time for same types - I mean different names for inner and outter fields. And everytime of course you can't find it in your cache dictionary, you save new record. If it's possible to fix - that'll be cool.

[StefH]

I will check your project.

However you can also fork this project and make a pull request on my source code : MemoryLeakTest project in scr-console folder.

[MakarovSoftware]

Ok, will do

[StefH]

I did quickly check your test code.
But this is not a bug, because you are creating different anonymous types (based on a random name)

See this example:

var list = new [] { 1, 2, 3 };
var q = list.AsQueryable();
var resultA = q.Select(x => new { A = x });
var resultB = q.Select(x => new { B = x });

The code above creates 2 different types.

System.Linq.Dynamic.Core will do the same, except it's dynamic.

[MakarovSoftware]

Yep, that's why I said that probably it's not a bug. But actually I can use same type in different parts of program, use different names and get this memory leak. And it's not a good idea. Maybe will be better to remove this string name from dictionary's key? Just leave only type names?

[StefH]

This sounds possible, I will take a look at the code and do some tests.

[StefH]

Sorry, this is not possible. Because the dynamic generated types do not only define the Type for each property, but off course the property-name itself is also generated.