MessageHub is holding objects in Memory

[czabransky]

The Subscriptions._localSubscriptions variable is populated during Publish, which creates a strong reference to handlers in that array. In the scenario where the thread is kept alive (eg, created on the UI thread), that collection is not updated until Publish is called again.

[ThreadStatic] private static Subscription[] _localSubscriptions;

In my code, that means MessageHub holds my object hierarchy in memory indefinitely, or until the next message is published. Is it possible to remove handlers from the _localSubscriptions during an UnRegister?

[NimaAra]

Okay, I understand your issue now. When calling UnRegister I am only removing the subscription from AllSubscriptions and not updating the _localSubscriptions therefore as you highlighted the subscription will remain in the variable until the next publish.

Okay I will fix this. Thank you for reporting it.

[czabransky]

Thanks Nima,

I am putting together an example if you still need it.

[NimaAra]

Yes that will be great as well.

Thanks.

[czabransky]

Here's a Console application that shows the issue:

internal class Program
{
    private static void Main(string[] args)
    {
        var recvrs = new List<MyReceiver>();
        for (var i = 0; i < 100; i++) {
            recvrs.Add(new MyReceiver());
        }

        PrintMemory("Initial");

        MessageHub.Instance.Publish(string.Empty);

        PrintMemory("1st Publish");

        foreach (var recvr in recvrs) {
            recvr.Dispose();
        }

        PrintMemory("Disposed");

        MessageHub.Instance.Publish(string.Empty);

        PrintMemory("2nd Publish");

        Console.ReadKey();
    }

    private static void PrintMemory(string step)
    {
        Console.WriteLine(step);
        Console.WriteLine($"Memory: {GC.GetTotalMemory(true)}");
        Console.WriteLine();
    }
}

public class MyReceiver : IDisposable
{
    private readonly Guid _token;

    public MyReceiver() => _token = MessageHub.Instance.Subscribe<string>(ReceiveMessage);

    private void ReceiveMessage(string msg)
    {
    }

    /// <inheritdoc />
    public void Dispose()
    {
        MessageHub.Instance.UnSubscribe(_token);
    }
}

[NimaAra]

Excellent, will fix it soon. Thanks.

[NimaAra]

Fixed the issue, please update the package and confirm.

[czabransky]

Hi Nima,

Works perfectly now! Thanks for the quick response, and thank you for your work on MessageHub. :)

[NimaAra]

Thank you for reporting it :-)

[nick1377]

This issue still occurs if publish is done on a different thread from the subscribe/unsubscribe. It is especially problematic if the publish is done in a Task that uses the thread pool since every time the publish is done it will be on a different thread and the subscription will be stored in the _localSubscriptions for that thread and never cleared leading to memory leaks.

[NimaAra]

Have you got a repro for me to look at?

[nick1377]

Sure. Pretty much the same as the above example except you execute the publish using Task.Factory.

internal class Program
{
    private static void Main(string[] args)
    {
        var recvrs = new List<MyReceiver>();
        for (var i = 0; i < 100; i++) {
            recvrs.Add(new MyReceiver());
        }
        PrintMemory("Initial");

        Task taskA = Task.Factory.StartNew(() => { MessageHub.Instance.Publish(string.Empty); });
        taskA.Wait();
        PrintMemory("1st Publish");
        foreach (var recvr in recvrs) {
            recvr.Dispose();
        }
        PrintMemory("Disposed");
        Console.ReadKey();
    }
    private static void PrintMemory(string step)
    {
        Console.WriteLine(step);
        Console.WriteLine($"Memory: {GC.GetTotalMemory(true)}");
        Console.WriteLine();
    }
}
public class MyReceiver : IDisposable
{
    private readonly Guid _token;
    public MyReceiver() => _token = MessageHub.Instance.Subscribe<string>(ReceiveMessage);
    private void ReceiveMessage(string msg)
    {
    } /// <inheritdoc /> 
    public void Dispose()
    {
        MessageHub.Instance.Unsubscribe(_token);
    }
}

[NimaAra]

The code above is not an accurate method of identifying leaks. It is measuring the amount of memory GC has allocated regardless of whether the GC was executed or not. A more accurate method would be:

private static void PrintMemory(string step)
{
    GC.Collect();
    GC.WaitForPendingFinalizers();
    Console.WriteLine(step);
    Console.WriteLine($"Memory: {GC.GetTotalMemory(true):N0}");
    Console.WriteLine();
}

I do not see any issue here. Did you run it in Debug or Release? Have you ran a memory profiler which can demonstrate the leak?

public static void Main()
{
    const int NUMBER_OF_SUBSCRIBERS = 100_000;
    
    long initial, afterPublish, afterDispose;            
    
    var recvrs = new MyReceiver[NUMBER_OF_SUBSCRIBERS];
    for (var i = 0; i < NUMBER_OF_SUBSCRIBERS; i++)
    {
        recvrs[i] = new MyReceiver(i);
    }

    initial = GetMemory();
    Console.WriteLine(nameof(initial) + ": " + initial.ToString("N0"));

    Task taskA = Task.Factory.StartNew(() => { MessageHub.Instance.Publish(string.Empty); });
    taskA.Wait();

    afterPublish = GetMemory();
    Console.WriteLine(nameof(afterPublish) + ": " + afterPublish.ToString("N0"));
    
    foreach (var recvr in recvrs)
    {
        recvr.Dispose();
    }

    afterDispose = GetMemory();
    Console.WriteLine(nameof(afterDispose) + ": " + afterDispose.ToString("N0"));
    Console.ReadLine();
}

private static long GetMemory()
{
    GC.Collect();
    GC.WaitForPendingFinalizers();
    return GC.GetTotalMemory(true);
}

private class MyReceiver : IDisposable
{
    private readonly Guid _token;
    public int Id { get; }

    public MyReceiver(int id)
    {
        _token = MessageHub.Instance.Subscribe<string>(ReceiveMessage);
        Id = id;
    }

    private void ReceiveMessage(string msg)
    {
    }

    public void Dispose() => MessageHub.Instance.Unsubscribe(_token);
}

[nick1377]

What I am trying to demonstrate is that it is holding the objects in memory when published on another thread.

For example if you run the same code without publishing in a task you get the following output.

private static void Main(string[] args)
{
    const int NUMBER_OF_SUBSCRIBERS = 100_000;
    var recvrs = new List<MyReceiver>();
    for (var i = 0; i < NUMBER_OF_SUBSCRIBERS; i++)
    {
        recvrs.Add(new MyReceiver());
    }
    PrintMemory("Initial");

    MessageHub.Instance.Publish(string.Empty);

    PrintMemory("1st Publish");
    foreach (var recvr in recvrs) {
        recvr.Dispose();
    }
    PrintMemory("Disposed");
    Console.ReadKey();
}

Initial
Memory: 18,963,624

1st Publish
Memory: 19,765,272

Disposed
Memory: 5,365,600

When you run it in the Task

private static void Main(string[] args)
{
    const int NUMBER_OF_SUBSCRIBERS = 100_000;
    var recvrs = new List<MyReceiver>();
    for (var i = 0; i < NUMBER_OF_SUBSCRIBERS; i++)
    {
        recvrs.Add(new MyReceiver());
    }
    PrintMemory("Initial");

    Task taskA = Task.Factory.StartNew(() => { MessageHub.Instance.Publish(string.Empty); });
    taskA.Wait();

    PrintMemory("1st Publish");
    foreach (var recvr in recvrs) {
        recvr.Dispose();
    }
    PrintMemory("Disposed");
    Console.ReadKey();
}

Initial
Memory: 18,963,624

1st Publish
Memory: 19,769,968

Disposed
Memory: 19,770,304

As you can see the objects aren't being disposed of when the publish is inside the task.

Because the publish and the unsubscribe are on different threads the unsubscribe can't remove the subscription from the [ThreadStatic] _localSubscriptions.

[NimaAra]

Yes you are right. The ThreadStatic was causing issues.

This should now be fixed (2nd time lucky!) in the latest version (v4.0.0).

Thanks for reporting this and let me know how it goes.

[nick1377]

I am still seeing a memory issue if you publish twice or more on separate threads before disposing.

internal class Program
{
    private static void Main(string[] args)
    {
        const int NUMBER_OF_SUBSCRIBERS = 100_000;
        var recvrs = new List<MyReceiver>();
        for (var i = 0; i < NUMBER_OF_SUBSCRIBERS; i++)
        {
            recvrs.Add(new MyReceiver());
        }

        PrintMemory("Initial");

        MessageHub.Instance.Publish(string.Empty);
        PrintMemory("1st Publish");

        Task taskA = Task.Factory.StartNew(() => {
            MessageHub.Instance.Publish(string.Empty);
        });

        taskA.Wait();

        PrintMemory("2nd Publish");

        foreach (var recvr in recvrs)
        {
            recvr.Dispose();
        }

        PrintMemory("Disposed");
        Console.ReadLine();
    }
    private static void PrintMemory(string step)
    {
        GC.Collect();
        GC.WaitForPendingFinalizers();
        Console.WriteLine(step);
        Console.WriteLine($"Memory: {GC.GetTotalMemory(true):N0}");
        Console.WriteLine();
    }
}
public class MyReceiver : IDisposable
{
    private readonly Guid _token;
    public MyReceiver() => _token = MessageHub.Instance.Subscribe<string>(ReceiveMessage);
    private void ReceiveMessage(string msg)
    {
    } /// <inheritdoc /> 
    public void Dispose()
    {
        MessageHub.Instance.Unsubscribe(_token);
    }
}

Initial
Memory: 18,964,288

1st Publish
Memory: 19,766,248

2nd Publish
Memory: 20,571,312

Disposed
Memory: 20,571,640

I believe the issue is just the break in UnRegister. Subscriptions can end up in multiple threads if you publish multiple times before disposing so it should always check all threads instead of stopping after it finds the first match.

[NimaAra]

Okay, let's give v4.0.1 a try?

[nick1377]

Looks to all be working now. No longer getting any memory leaks. Thanks for the quick responses!

[NimaAra]

Excellent, once again, thank you for reporting.