[Bug] CollectionView adding items - items jumping erratically

[Tommigun1980]

I tried converting my bindable stack layouts to collection views but am running into several problems again preventing me from using them.

I have items in the collection view that are of different sizes, so ItemSizingStrategy is MeasureAllItems. I have a LinearItemsLayout with ItemSpacing. The collection view serves as container for chat messages.

No matter what ItemsUpdatingScrollMode is set to, items and the scroll position jumps around randomly when I add new items. This error happens when the CollectionView's item measurement strategy has been set to MeasureAllItems, items of different heights are visible, and a new element is added.

Video of the issue with a CollectionView, with MeasureAllItems:
https://drive.google.com/file/d/1avaPs2O0oSQNHBEG5GpExNawNB9QvVD5/view?usp=sharing

Video of the issue with a ListView, with UnevenRows:
https://drive.google.com/file/d/1yoKMVuCU0Rerp0evjZ-N8X4c7nbJLmDq/view?usp=sharing

XF 5 latest preview, iOS (didn't test on Android).

EDIT: Added videos.
Please note: A comment further down this thread has repro steps.

[acuntex]

If you implement the Scrolled event and track e.VerticalOffset you can see that during the scroll on iOS the offset increases and suddenly jumps to 0, and then back.

It's totally messed up, if you want to show a "Scroll Up"-Button depending on the Scroll State, you can see the Button appearing/disappearing because there is suddenly this "0".

[Tommigun1980]

Here is a video of the issue with a CollectionView:
https://drive.google.com/file/d/1avaPs2O0oSQNHBEG5GpExNawNB9QvVD5/view?usp=sharing

Here is a video of the issue in a ListView:
https://drive.google.com/file/d/1yoKMVuCU0Rerp0evjZ-N8X4c7nbJLmDq/view?usp=sharing

What is weird is that any CONSECUTIVE add operations work just fine in a ListView:
https://drive.google.com/file/d/1HOFfEohNkT3R8uTctkEHu80VYcWLk-aQ/view?usp=sharing
(But not in a CollectionView).

If I restart the app, then it always jumps once when adding items.

Couple of notes:

1. This happens only when measurement strategy is all items, or in a list view if uneven rows is used.
2. ListView: This happens only one time when adding a new item. CollectionView: This happens every time the collection is edited.
3. This happens only if I scroll to the newly added item.
4. The above videos were in fact taken from a list view, as the effect is exactly the same and I tried converting the collection views to list views (but the same problem happens there).

So final repro steps:

1. Have a collection view or list view with uneven rows/measure first item.
2. Add a new item to it.
3. Scroll to the newly added item very quickly after adding a new item (I have a listener that gets invoked when a new item is available, which updates the VM's ObserverableCollection and asks the list to scroll to the last item in the ObserverableCollection)
4. The issue happens.
5. Add more items - issue doesn't happen anymore.

This issue doesn't happen with a bindable StackLayout, even when scrolling to newly added items.
I think this issue is related to #12554 which may be caused by #11511 and #10842.

So I think all of these issues have to do with bugging measurements of items, possibly when using data templates of different heights (or merely if the items themselves have different heights, haven't tested).

If this could be fixed CollectionView would become the de-facto collection component, but unfortunately I can't use it at all with these issues. The fact ListView suffers of this same exact problem makes me unable to ship anything. Please please prioritise this as it completely defeats the purpose of the collection and list views as you can't alter them.

I did find a partial workaround though -- adding an arbitrary delay between the list view reading a new item from its items source, and scrolling to it, makes the issue not appear. Problem is I have no idea whatsoever how long this delay should be and is surely device power and list length/complexity dependent, so it's not a good workaround.

[Tommigun1980]

I went back to a ListView for now as it works slightly better than a collection view. This is my code just for jumping to the last message + hiding the list jumping around on first insertion. Three hacks for the price of one:

        private void ScrollToLastMessage(bool animate)
        {
            if (this.vm.ChatMessages.Count > 0)
            {
                var lastItem = this.vm.ChatMessages[this.vm.ChatMessages.Count - 1];
                this.chatMessagesListView.ScrollTo(lastItem, ScrollToPosition.End, animate);
            }
        }

        private async void ChatMessagesListView_ItemAppearing(object sender, ItemVisibilityEventArgs e)
        {
            // TEMP HACK: can't call ScrollToLastMessage when the list has been populated
            // as it may miss a few items at the tail end. so wait until at least one item
            // has appeared
            if (this.tempHackIsInitialScrollToLastMessageRequested)
            {
                this.tempHackIsInitialScrollToLastMessageRequested = false;
                this.ScrollToLastMessage(false);
            }

            if (e.ItemIndex == this.vm.ChatMessages.Count - 1)
            {
                this.isLastChatMessageVisible = true;

                // TEMP HACK! even when an item is claimed to be visible (ItemAppearing), it is not so delay a bit.
                // we jump to the last message when data has been loaded so we need an overlay hiding the list,
                // but even when ItemAppearing claims the last item has been made visible the list still shows the
                // initial messages... so delay a slight bit here.
                if (!this.HasFullyLoadedChatMessages)
                {
                    await Task.Delay(100); // this needs to be increased the more complex the layout is. why does it claim an item has appeared when it has not?
                    this.HasFullyLoadedChatMessages = true;
                }
            }
        }

        private async void OnChatMessageReceived(IChatService.OnChatMessageReceivedParams args)
        {
            if (this.isLastChatMessageVisible)
            {
                // TEMP HACK! for the first new item added to a list, and if it's scrolled
                // near its insertion time, the list jumps around.
                if (!this.tempHackHasReceivedAtLeastOneNewMessage)
                {
                    await Task.Delay(200);
                    this.tempHackHasReceivedAtLeastOneNewMessage = true;
                }

                this.ScrollToLastMessage(true);
            }
        }

StackLayout at least has ResolveLayoutChanges() that would allow me to remove some of the hacks, but alas, there is no such thing in ListView or CollectionView. I was using it when I had a bindable StackLayout before foolishly trying to adapt a virtualising component.

Edit: Doesn't work on Android so I'm stuck again :/

[Tommigun1980]

I'd like not having to use a ListView, but the original problem with the CollectionView stands: it doesn't work if items have different sizes when new elements are added as things jump around like crazy... so I'm stuck with the ItemList implementation as I don't think that one is solvable without Xamarin fixing it.

So if one thing gets fixed in Xamarin.Forms I hope it's MeasureAllItems in collection view!!!

If these issues are hard to repro I would be willing to give access to my project (after an appropriate NDA has been signed), but I assume the issues are exposed with MeasureAllItems and any kind of content. If not please ping me.

[Tommigun1980]

If you implement the Scrolled event and track e.VerticalOffset you can see that during the scroll on iOS the offset increases and suddenly jumps to 0, and then back.

It's totally messed up, if you want to show a "Scroll Up"-Button depending on the Scroll State, you can see the Button appearing/disappearing because there is suddenly this "0".

This seems to be the root cause. It is very much visible with a CollectionView inside a RefreshView as per #12622, and I also debugged that what happens when new elements are added to a CollectionView is that ItemAppearing is called for the last element, but then ItemDisappearing is called for it directly afterwards, then ItemAppearing for a few earlier elements, and ultimately ItemAppearing for the last item AGAIN. So it is exactly as you say, it seems to do an internal scroll whenever the collection changes when MeasureAllItems is turned on and items of different heights are visible (and the same for a ListView with UnevenRows turned on). This is also the reason why I have to add arbitrary delays and and an overlay on top of the view while it's bouncing around, and can't trust the events.

[acuntex]

@Tommigun1980 Do you somehow use FFImageLoading?

I replaced FFImageLoading with Xamarin.Forms.Nuke/GlideX.Forms and the scrolling problems are gone now.

[Tommigun1980]

@Tommigun1980 Do you somehow use FFImageLoading?

I replaced FFImageLoading with Xamarin.Forms.Nuke/GlideX.Forms and the scrolling problems are gone now.

Wow, I do! Thank you, I will look into it. Maybe all the issues have been caused by FFImageLoading then? I will test Glide/Nuke and report back, but I am unsure if it will fit my needs. I have a very delicate caching strategy where some things are cached on disk + memory, while the bulk only on disk. Glide/Nuke doesn't seem to have any control over it, and the docs don't even mention if "cache" means memory or disk cache. I don't want the clients to redownload all the assets on every boot and only cache them in memory if that's all Glide/Nuke does (I'm paying for the CDN also :P). In addition to that I need to be able to preload images, and at least the main doc page for Nuke doesn't mention any such functionality but I'll dig into it.

Anyhow, I'll give it a try and see if FFImageLoading is actually the one creating all of these problems. Thanks for the info, mate!

[acuntex]

You can also check https://github.com/kean/Nuke/

If the wrapper provided with Xamarin.Forms.Nuke exposes all APIs then it's possible to set the Cache strategy.

Just in case: If you use glidex, you have to provide the HeightRequest/WidthRequest of the images because in some cases it does not show any image if you don't provide it.

[Tommigun1980]

You can also check https://github.com/kean/Nuke/

If the wrapper provided with Xamarin.Forms.Nuke exposes all APIs then it's possible to set the Cache strategy.

Just in case: If you use glidex, you have to provide the HeightRequest/WidthRequest of the images because in some cases it does not show any image if you don't provide it.

Unfortunately the XF integration seems to be platform specific and gives no control on the XF side. I will try temporarily just removing the FFImageLoading to see if it really is causing all these problems, but I do see problems in CollectionViews where FFImageLoading images reside where they have a height set. So I don't see how they could break the collection/list views and cause them to jump up and down? And everything is fine as long as rows have even heights so I really doubt it's caused by FFImageLoading?

[Tommigun1980]

Yeah I confirmed that the issues happen without FFImageLoading. But thanks for the tip anyway, I really appreciate it!!

[acuntex]

@Tommigun1980
Yes, in order to configure Nuke or GlideX you would have to put the configuration code for each platform separate in AppDelegate/MainActivity

Have you removed FFImageLoading completely and you still have the scrolling issues? (Not only replacing CachedImage with Image with CachedImageRenderer.InitImageSourceHandler() active, but the complete NuGet package)

I do have different heights (basically like WhatsApp with messages/images) and the problem is solved.
Btw. you should also try the fix described here: #10842 (comment)
It's a custom renderer for CollectionView that fixes sizing problems with different heights.

[Tommigun1980]

@Tommigun1980
Yes, in order to configure Nuke or GlideX you would have to put the configuration code for each platform separate in AppDelegate/MainActivity

Have you removed FFImageLoading completely and you still have the scrolling issues? (Not only replacing CachedImage with Image with CachedImageRenderer.InitImageSourceHandler() active, but the complete NuGet package)

I do have different heights (basically like WhatsApp with messages/images) and the problem is solved.
Btw. you should also try the fix described here: #10842 (comment)
It's a custom renderer for CollectionView that fixes sizing problems with different heights.

Hi. I don't see any issue with FFImageLoading so I won't be replacing it but thanks for the suggestion anyway! I am positive it has nothing to do with the collection view and list view issues as I had issues with them before even using FFImageLoading, and I can't see how just the mere presence of its NuGet would somehow break the other components. But I did have the entire NuGet removed when I tested removing the images from the collection and list views and the issues persisted. It isn't related to it.
I will try the renderer hacks in #10842 (comment), thanks again for trying to help me, I really appreciate it!

[Tommigun1980]

@acuntex: The custom renderers in the threads seem all to be lacking to some degree and either try to hard code the heights of the data templates (can't do as I'm displaying chat messages), or seem to break something else like grouping or headers. If someone provides a renderer that just fixes this issue without breaking something else I'd definitely try it.

[Tommigun1980]

Hi @jsuarezruiz. I saw that you had assigned yourself to some of the collection view issues. After having a look at the open issues it seems as this report is a duplicate. In fact there's a ton of reports that seem to all have to do with this exact same measurement problem in CollectionViews (and ListViews), that are exposed when measurement strategy is set to all items and items of different heights are visible. The issues manifest when adding items, when scrolling etc.

Do you think there is any chance this could get fixed at some point, as I am honestly unsure of how to proceed. I have spent so much time on this and it all comes back to this issue. I wouldn't be surprised if almost all of my reports had the same root cause of incorrect measurements.

Thank you so much.

[Tommigun1980]

Repro project at #13231 (bugs 3 and 4).

[brentedwards]

I see a similar issue with CollectionView. We have implemented forever scroll functionality in our app. When the user hits the bottom of the items loaded, the next chunk is loaded. If I scroll to the bottom fast enough, I can see the next record added. However, once I scroll, the CollectionView jumps up the items and it looks as though the app is loading duplicate records. As the CollectionView gets filled with more and more items, that jump can become rather significant. Our application uses Xamarin.Forms 4.5.0.657, however I was able to reproduce this issue in a standalone sample app which uses the latest stable release (5.0.0.1874). I've attached a zip of that sample app.
CollectionViewSample.zip

[Tommigun1980]

I see a similar issue with CollectionView. We have implemented forever scroll functionality in our app. When the user hits the bottom of the items loaded, the next chunk is loaded. If I scroll to the bottom fast enough, I can see the next record added. However, once I scroll, the CollectionView jumps up the items and it looks as though the app is loading duplicate records. As the CollectionView gets filled with more and more items, that jump can become rather significant. Our application uses Xamarin.Forms 4.5.0.657, however I was able to reproduce this issue in a standalone sample app which uses the latest stable release (5.0.0.1874). I've attached a zip of that sample app.
CollectionViewSample.zip

Same happens when items are deleted.

[hartez]

@Tommigun1980 are you still seeing this problem with 5.0.0.1931?

@brentedwards I tried to run your repro app, but it just crashes on load.

[Tommigun1980]

@Tommigun1980 are you still seeing this problem with 5.0.0.1931?

@brentedwards I tried to run your repro app, but it just crashes on load.

Hi @hartez! I can see the issue partially in 5.0.0.1931 when adding items, the CollectionView is using 'KeepLastItemInView' and it is not scrolled to the bottom. I just replied to you here regarding it:
#13231 (comment)

It is reproducible with the 'Mega repro project for CollectionView issues' and I added a new video there describing the problem.

Thank you!

[Tommigun1980]

(This is an old ticket wherefrom I extracted the bugs into the 'Mega repro project for CollectionView issues' ticket and repro project, which also has more information and better videos of the issues. So maybe this ticket should be closed?

This ticket does contain information about similar ListView problems, but considering it's been deprecated in favour of the CollectionView, and the CollectionView received such solid updates fixing a lot of the pressing issues at least I wouldn't mind if the ListView stayed as it is).

Thank you again.

[Tommigun1980]

@hartez I'll go ahead and close this ticket in favour of #13231. If you feel differently please reopen it!