[Feature] Be able to read the gallery as fast as the native IOS

[Tom3652]

Thanks for the API !

Is your feature request related to a problem? Please describe.

When you scroll down in the IOS native gallery, you never see any jank or blinking for any photo.
Not sure the issue is related to the plugin though, but i am loading thumbnails from memory and they are all blinking on both Android and IOS with white placeholders.
I have checked the performances of retrieving assets and it seems very fast actually, the problem doesn't come from the retrieval of assets but rather in UI.

Describe the solution you'd like
If you know how to load assets in a way that everything seems always preloaded, you could make a simple Widget that takes care of this.
If you want to let this API be without UI, can you provide in the documentation an example of what you would do in your app to have a similar gallery as the native IOS (not talking about Staggered grid, but simply removing blinking) ?

Describe alternatives you've considered
If this is not possible with the plugin, i am considering doing it natively (Swift) because it works perfectly on IOS (and i have 5K+ photos / videos on my phone, without any blinking and even with a very good resolution)
I guess the same on Android.
But again, the problem is a UI problem, not from the plugin but since i can't find a solution with Flutter Widgets...

[AlexV525]

See fluttercandies/flutter_wechat_assets_picker#116 .

[jaysignorello]

@Tom3652 It all depends on the implementation as I've learned the hard way. If you could share any code, it'd be helpful to see if we could give you any pointers on how to make it faster.

[Tom3652]

@jaysignorello thank you for your reply and your help in advance, here is the code i am using to display the assets :

class GalleryThumbnail extends StatelessWidget {

  int index;
  final AssetEntity asset;
  
  GalleryThumbnail({required this.asset, required this.index});

  @override
  Widget build(BuildContext context) {
    return FutureBuilder<Uint8List?>(
      future: Platform.isIOS ? asset.thumbDataWithOption(
        ThumbOption.ios(
          width: 500,
          height: 500,
          deliveryMode: DeliveryMode.opportunistic,
          resizeMode: ResizeMode.fast,
          resizeContentMode: ResizeContentMode.fit,
          quality: 100
          // resizeContentMode: ResizeContentMode.fill,
        ),
      ) : asset.thumbDataWithSize(250, 250),
      builder: (_, snapshot) {
        final bytes = snapshot.data;
        if (snapshot.hasError) {
          return Container();
        }
        // If we have no data 
        if (bytes == null) return Container();
        // If there's data, display it as an image
        return ExtendedImage.memory(bytes, fit: BoxFit.cover);
      },
    );
  }
}

These items are displayed in a simple GridView.builder().

The result is the same if i use Image.memory() instead of ExtendedImage.memory(), I can see flickering and white blinks as i scroll down the list

500 seems high for a thumbnail but on IOS the quality is really lowered if it's not set to such value, compared to the IOS Gallery once again

[jaysignorello]

@Tom3652 thanks, that's helpful.

Likely, the reason for the flickering is because the GalleryThumbnail widget is getting rebuilt as you scroll. Somethings to consider on the optimizations front.

1. Make sure you're not rebuilding the entire view unnecessary, such as calling setState unless you absolutely need to rebuild. I've seen folks calling setState in odd places.
2. Use gaplessPlayback: true on Image/ExtendedImage
3. Use file caching of thumbnails if you need to resize for higher quality. You're right that iOS needs a higher quality due to pixel density. Instead of picking 500x500, you can take the size of the image you need (say for example 75x75) and multiple it by the phones pixel density. This way you're not making unnecessary large photos.
4. Reduce the quality of the thumbnail. Along with using the lowest thumbnail size, reducing the quality will help you keep more images in memory, which will increase performance.
5. Another trick you can use is make a stateful widget that initially calls asset.thumbData, which is a lower quality version of the file. This call is usually very fast, which can be used as a loading image that will give the user at least some idea of what is to come as you generate the higher quality photo.
6. Preload the caching so future requests are fast. This library does have an experimental feature to do this, but you can also roll your own.

PhotoCachingManager().requestCacheAssets(
  assets: assets,
  option: thumbOption,
);

[Tom3652]

Thanks for the detailed tips !

1. I am using the provider package and i am reloading the list (notifyListeners()) only every 50 assets, i don't load them all in ont shoot but as long as i am scrolling i fetch new assets by range with something like this :

Future<void> onLoadMore(int lastIndex) async {
    if(!fetching) {
      fetching = true;
      if (showItemCount >= album.assetCount) {
        print("already max");
        return;
      }

      //final list = await album.getAssetListPaged(page + 1, 50);
      final list = await album.getAssetListRange(start: lastIndex, end: lastIndex+50);
      //page += 1;
      this.assetsList.addAll(list);
      notifyListeners();
      fetching = false;
      //printListLength("loadmore");
    }
  }

2. gaplessPlayback: true seems to have removed flickering !
3. I will sound stupid, but i don't really understand how can i multiply by the pixel density... Is it the widget size or the thumbnail ? If you can provide me a quick example i will be very thankful !
4. I will lower the quality you are right
5. About caching, how does actually works ? i simply need to call it once after the first time i have fetched the assets ?

Thanks again for your time

[jaysignorello]

@Tom3652 Glad to hear that gaplessPlayback: true has helped to fix the problem.

There is never a stupid question. The size you'll be looking to calculate is the thumbnail. If say your using a grid and each grid item is 75x75, you'll need to increase the size of the thumbnail to match the devicePixelRatio of the device. Otherwise you get the poor looking images.

To accomplish that, you'll get devicePixelRatio and make the calculation like so.

  Size gridSize = Size(75,75);
  double devicePixelRatio =  MediaQuery.of(context).devicePixelRatio;
  Size thumbSize = Size(gridSize.width * devicePixelRatio, gridSize.height * devicePixelRatio)

  ...

  GalleryThumbnail(asset: asset, index: 1, thumbSize: thumbSize)

Regarding the caching, you'd call it before you'd render the assets for the user. You won't want to walk through their entire library and precache ever image. Instead for example, as the user is scrolling, you might pull the assets earlier than you are now, then kick off doing the precache, and start loading the assets into the UI. This is obviously advanced and will require a good deal of work to get right, but it's most certainly what Apple is doing to help make their experiences smooth.

[Tom3652]

Thanks again @jaysignorello it helps a lot for performances !
I have even made a thumbnail in lower quality and size than asset.thumbData.

I will try to implement the caching system. So far, it's already much better than what i had, but if i scroll very fast i still have blank space for like 200ms the time the FutureBuilder get the thumbnail and loads it into the Image.memory()```.

Here is the "normal output" :

I am trying to do something like this with the cache system you provide :

final caching = await album.getAssetListRange(start: lastIndex+50, end: lastIndex + 100);
PhotoCachingManager().requestCacheAssets(assets: caching, 
          option: ThumbOption.ios(width: (75 * Constants.devicePixelRatio).floor(),
          height: (75 * Constants.devicePixelRatio).floor(), quality: 80));
final list = await album.getAssetListRange(start: lastIndex, end: lastIndex+50);
this.assetsList.addAll(list);
notifyListeners();

The return of PhotoCachingManager().requestCacheAssets(assets: assets); is Future<void>, i was hoping to be able to retrieve some list"of Uint8list thumbnails that i can use directly in the GalleryThumbnail Constructor instead of using the FutureBuilder.
I would like in the above code to change the list into the list of cached thumbnails as Uint8list and not as AssetEntity
Am i missing something here about caching ?
I am aware these APIs are experimental as stated in the doc and as you told me of course, so in the meantime i have implemented my own system that is working but needs a lot of improvements, it is just to show you :

I am storing myself the Uint8list such as :

int width = (Constants.maxWidth / 3 * Constants.devicePixelRatio).floor();
int height = (Constants.maxWidth / 3 * 2).floor();
HashMap<String, Uint8List> assetsCached = HashMap();

Future<void> getMemoryThumbFromAsset(List<AssetEntity> list) async {
    for (AssetEntity entity in list) {
      if (!assetsCached.containsKey(entity.id)) {
        Uint8List? asset = await entity.thumbDataWithOption(
          ThumbOption.ios(
              width: width,
              height: height,
              deliveryMode: DeliveryMode.opportunistic,
              resizeMode: ResizeMode.fast,
              resizeContentMode: ResizeContentMode.fit,
              quality: 80
            // resizeContentMode: ResizeContentMode.fill,
          ),
        );
        if (asset != null) {
          assetsCached[entity.id] = asset;
        }
      }
    }
  }


Future<void> onLoadMore(int lastIndex) async {
    if(!fetching) {
      fetching = true;
      if (showItemCount >= album.assetCount) {
        print("already max");
        return;
      }
      //final list = await album.getAssetListPaged(page + 1, 50);
      /*
      PhotoCachingManager().requestCacheAssets(assets: caching,
          option: ThumbOption.ios(width: (75 * Constants.devicePixelRatio).floor(),
          height: (75 * Constants.devicePixelRatio).floor(), quality: 80));

       */
      final list = await album.getAssetListRange(start: lastIndex, end: lastIndex+100);
      await getMemoryThumbFromAsset(list);
      this.assetsList.addAll(list);
      notifyListeners();
      fetching = false;
      printListLength("loadmore");
    }
  }

Here is the result :

I will do better maths to avoid having 5k+ thumb in memory, but around 200-300 all the time by recycling the assetsCached.
Maybe later the caching system will do something equivalent and return a list of cached thumbs with a specified item ranged to keep 🚀

[jaysignorello]

@Tom3652 That's great.

I would like in the above code to change the list into the list of cached thumbnails as Uint8list and not as AssetEntity
Am i missing something here about caching ?

PhotoCachingManager().requestCacheAssets(...) is meant to generate the thumbnails on disk for faster retrieval in the future, which is a typical caching strategy.

While your approach might work alright in a non-release/dev setting, I wouldn't recommend it. The problem is that you need to be careful about your apps memory usage and make sure it's a good citizen. iOS for example will kill your app if it's consuming too much memory, so it's important to be mindful of that.

For caching implementation, I'd suggest checking out existing solutions such as https://pub.dev/packages/flutter_cache_manager or if you want to roll your own, adopting a LRU cache implementation so you're limiting the number of objects in memory at once to ones that have been recently used.

[jaysignorello]

@Tom3652 and oh, seems like we can close out this ticket now.

[Tom3652]

Thanks again for the useful links and the help, yes i am aware about the memory issue and will not release something like this.
Disk cache strategy seems the good way to go
Yes indeed we can close it now :)

[AlexV525]

Thanks for all these solid solutions for the performance.