Dynamic Filter with SourceList

[Liklainy]

Hello!
I am new to DynamicData and cannot understand why there is no update after removing the items from filtered SourceList.

using DynamicData;
using System;
using System.Collections.Generic;
using System.Linq;
using System.Reactive.Linq;
using System.Reactive.Subjects;

namespace DynamicDataTests
{
    class UserPermission
    {
        public int Id;
        public int UserId;
        public int RoleId;
    }

    class Program
    {
        static void Main(string[] args)
        {
            var subject = new Subject<HashSet<int>>();
            var dynamicFilter = subject.Select(ids => (Func<UserPermission, bool>)(x => ids.Contains(x.UserId)));

            var list = new SourceList<UserPermission>();

            list.Connect()
                //.Filter(dynamicFilter)
                .ToCollection()
                .Subscribe(x => Console.WriteLine($"Changed: {string.Join(",", x.Select(up => up.Id))}"));

            list.Edit(innerList =>
            {
                innerList.Add(new UserPermission { Id = 0, UserId = 0, RoleId = 0 });
                innerList.Add(new UserPermission { Id = 1, UserId = 0, RoleId = 0 });
                innerList.Add(new UserPermission { Id = 2, UserId = 0, RoleId = 1 });
                innerList.Add(new UserPermission { Id = 3, UserId = 0, RoleId = 1 });
                Console.WriteLine($"Items in collection: {innerList.Count}");
            });

            subject.OnNext(new HashSet<int>(new[] { 0 }));

            list.Edit(innerList =>
            {
                var toRemove = innerList.Where(x => x.RoleId == 1 && x.UserId == 0);
                Console.WriteLine($"Remove: {string.Join(",", toRemove.Select(up => up.Id))}");
                innerList.RemoveMany(toRemove);
                Console.WriteLine($"Items in collection: {innerList.Count}");
            });

            Console.ReadKey();
        }
    }
}

What I got:

Items in collection: 4
Changed: 0,1,2,3
Remove: 2,3
Items in collection: 2

What I expected to see:

Items in collection: 4
Changed: 0,1,2,3
Remove: 2,3
Items in collection: 2
Changed: 0,1

Why there is no update after removing items?

[RolandPheasant]

It looks to be working for me. ToCollection() materialises inner contents and does not report changes. For the actual changes you need to remove this operator.

[RolandPheasant]

I missed the filter part of the question. Use a behavior subject and pass an initial filter in. It should work then

[Liklainy]

@RolandPheasant, Thank you, for your fast answer!
Sorry, that i forgot to uncomment the filter.
Yes, with BehaviorSubject it works as expected.

In my project I use ReactiveUI's WhenAnyValue
And I have the next similar problem with it

using DynamicData;
using ReactiveUI;
using System;
using System.Collections.Generic;
using System.Linq;
using System.Reactive.Linq;

namespace DynamicDataTests
{
    class UserPermission
    {
        public int Id;
        public int UserId;
        public int RoleId;
    }

    class FilterContainer : ReactiveObject
    {
        private HashSet<int> _filter;

        public HashSet<int> Filter
        {
            get => _filter;
            set => this.RaiseAndSetIfChanged(ref _filter, value);
        }
    }

    class Program
    {
        static void Main(string[] args)
        {
            var filterContainer = new FilterContainer();
            filterContainer.Filter = new HashSet<int>();

            var dynamicFilter = filterContainer
                .WhenAnyValue(x => x.Filter)
                .Select(ids => (Func<UserPermission, bool>) (x => ids.Contains(x.UserId)));

            var list = new SourceList<UserPermission>();

            list.Connect()
                .Filter(dynamicFilter)
                .Subscribe(x =>
                {
                    Console.WriteLine($"Changes: {x.TotalChanges}");
                });

            list.Connect()
                .Filter(dynamicFilter)
                .ToCollection()
                .Subscribe(x =>
                {
                    Console.WriteLine($"Items: {string.Join(",", x.Select(up => up.Id))}");
                });

            Console.WriteLine("Add items to list");

            list.Edit(innerList =>
            {
                innerList.Add(new UserPermission {Id = 0, UserId = 0, RoleId = 0});
                innerList.Add(new UserPermission {Id = 1, UserId = 0, RoleId = 0});
                innerList.Add(new UserPermission {Id = 2, UserId = 0, RoleId = 1});
                innerList.Add(new UserPermission {Id = 3, UserId = 0, RoleId = 1});
                Console.WriteLine($"Items in collection: {innerList.Count}");
            });

            Console.WriteLine("\nShow User 0");
            filterContainer.Filter = new HashSet<int>(new[] {0});

            Console.WriteLine("\nRemove items from list");
            list.Edit(innerList =>
            {
                var toRemove = innerList.Where(x => x.RoleId == 0);
                Console.WriteLine($"Remove: {string.Join(",", toRemove.Select(up => up.Id))}");
                innerList.RemoveMany(toRemove);
                Console.WriteLine($"Items in collection: {innerList.Count}");
            });

            Console.WriteLine("\nShow User 1");
            filterContainer.Filter = new HashSet<int>(new[] {1});

            Console.ReadKey();
        }
    }
}

Output is

Add items to list
Items in collection: 4

Show User 0
Changes: 4
Items: 0,1,2,3

Remove items from list
Remove: 0,1
Items in collection: 2

Show User 1
Changes: 2
Items: 0,1

There are no updates when items was removed.
And for UserId 1 it returned items which was removed from the list.

I think that there are problems with threads/synchronization.
Can you suggest something?

[RolandPheasant]

I am not sure. Is the changed event always firing, and does .WhenAnyValue(x => x.Filter) give you an initial value?

[glennawatson]

WhenAnyValue triggers straight away with the default value of the property, then will trigger subsequent times. If you don't want WhenAnyValue()' to do the initial value, you can use .Skip(1)`

[Liklainy]

Changed event is always firing for the not filtered list.

WhenAnyValue is similar to WhenValueChanged in NotifyPropertyChangedEx
So I added Remove part to DynamicFilter test in DynamicData.Snippets project:

 DynamicData.Snippets/Filter/FilterFixture.cs | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/DynamicData.Snippets/Filter/FilterFixture.cs b/DynamicData.Snippets/Filter/FilterFixture.cs
index d388511..c6c387e 100644
--- a/DynamicData.Snippets/Filter/FilterFixture.cs
+++ b/DynamicData.Snippets/Filter/FilterFixture.cs
@@ -78,6 +78,13 @@ namespace DynamicData.Snippets.Filter
                 schedulerProvider.TestScheduler.Start();
                 sut.Filtered.Items.ShouldAllBeEquivalentTo(_items.Where(a => a.Type == "Frog"));
                 sut.Filtered.Count.Should().Be(1);
+
+                //Remove
+                sourceList.RemoveMany(_items.Where(x => x.Type == "Frog"));
+                sut.AnimalFilter = "Whale";
+                schedulerProvider.TestScheduler.Start();
+                sut.Filtered.Items.ShouldAllBeEquivalentTo(_items.Where(a => a.Type == "Whale"));
+                sut.Filtered.Count.Should().Be(1);
             }
         }

And it works...
Until you provide an initial filter which will result in an empty filtered list.

 DynamicData.Snippets/Filter/FilterFixture.cs | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/DynamicData.Snippets/Filter/FilterFixture.cs b/DynamicData.Snippets/Filter/FilterFixture.cs
index c6c387e..9e20a2c 100644
--- a/DynamicData.Snippets/Filter/FilterFixture.cs
+++ b/DynamicData.Snippets/Filter/FilterFixture.cs
@@ -49,12 +49,17 @@ namespace DynamicData.Snippets.Filter
             using (var sourceList = new SourceList<Animal>())
             using (var sut = new DynamicFilter(sourceList, schedulerProvider))
             {
-                //start the scheduler
+                //set initial filter
+                sut.AnimalFilter = "Kitty";
                 schedulerProvider.TestScheduler.Start();
 
                 //add items
                 sourceList.AddRange(_items);
 
+                //clear filter
+                sut.AnimalFilter = null;
+                schedulerProvider.TestScheduler.Start();
+
                 sut.Filtered.Items.ShouldAllBeEquivalentTo(_items);
                 sut.Filtered.Count.Should().Be(_items.Length);

The test fails, because in the filtered list there is the removed Frog

[Liklainy]

diff.txt
I added a test which shows an incorrect behavior of the filtered list:

[Fact]
public void RemoveFiltered()
{
    var person = new Person("P1", 1);

    _source.Add(person);
    _results.Data.Count.Should().Be(0, "Should be 0 people in the cache");
    _filter.OnNext(p => p.Age >= 1);
    _results.Data.Count.Should().Be(1, "Should be 1 people in the cache");

    _source.Remove(person);

    _results.Data.Count.Should().Be(0, "Should be 0 people in the cache");
}

In the beginning there is a filter Age > 20, so when added there should be no people in the result.
After applying filter Age >= 1 there is a added person in the list.
But after removing from the source list the filtered result is not updated.
So the test with the current implementation will fail on the last assertion.

I think there is an error in the internal state of Filter.