Fast scrolling with a lot of images causes excessive memory usage

[kerrmarin]

Hi,

I have an iPad app where I show rows of 4 (relatively large: width: 920px; height: 1380px;) images on an iPad inside a collection view. This leads to, when scrolling moderately rapid, excessive memory usage and eventually a crash (on physical devices). An example instruments call tree:

An example of memory usage over time:

Here is a device log from the crash:

Incident Identifier: 9C9452D1-8376-4E06-8F2F-8E6F26080653
CrashReporter Key:   34a30a2fec33ac19197705866e1020fc84df87bf
Hardware Model:      iPad2,5
OS Version:          iPhone OS 7.0.4 (11B554a)
Kernel Version:      Darwin Kernel Version 14.0.0: Fri Sep 27 23:00:49 PDT 2013; root:xnu-2423.3.12~1/RELEASE_ARM_S5L8942X
Date:                2015-09-30 12:45:46 +0100
Time since snapshot: 125 ms

Free pages:                              1182
Active pages:                            3665
Inactive pages:                          1845
Speculative pages:                       266
Throttled pages:                         102067
Purgeable pages:                         0
Wired pages:                             19868
File-backed pages:                       5433
Anonymous pages:                         343
Compressions:                            0
Decompressions:                          0
Compressor Size:                         0
Uncompressed Pages in Compressor:        0
Largest process:   PINRemoteTest

Processes
     Name                    <UUID>                       rpages       recent_max   fds      [reason]          (state)

   wirelessproxd <46066fc432663d45ab7e055082fe0bd6>          139              139  200   [vm-pageshortage]  (daemon) (idle)
      MobileMail <b3574f4bded1315cb2e50e5de205be48>         1137             1137  200   [vm-pageshortage]  (resume) (continuous)
            tccd <1fea8c5a71943151b5cd304c7eb0fd8c>          160              160  100   [vm-pageshortage]  (daemon)
   PINRemoteTest <8aac78eadffc37b0ac68272860f87788>        74944            74944  200   [vm-pageshortage]  (frontmost) (resume)
            ptpd <db9048c36f6c3c18a7330fc96d93a0cf>          647              647   50                      (daemon)
identityservices <18cc20db2e4739a782cc8e38e03eff52>          364              364  100                      (daemon)
           wifid <a5cf99e5a0f032a69bc2f65050b44291>          379              379   25                      (daemon)
         syslogd <6539f4cf4dcf34daadf1d99991926680>          155              166   50                      (daemon)
       locationd <c31643022d833911b8b7461fd3964bd5>          955              955  100                      (daemon)
          powerd <0a253ac2a99236809422214be1700bc0>          118              118  100                      (daemon)
         imagent <bef102e1faef39209926fb25f428a71e>          240              240  100                      (daemon)
   iaptransportd <42faa147f61a314bb735e239f445efaf>          205              205   50                      (daemon)
    mediaserverd <71101024312538ccae5799b88681e38d>          667              667  100                      (daemon)
   mDNSResponder <8922e9954d893eb9a1ab27ca4723bbab>          210              280  100                      (daemon)
            apsd <0dd1fd7c2edc3cf9899a4830541c1bac>          437              437  100                      (daemon)
     dataaccessd <5da732b6ce6935f2928461a022101aba>          941              941  100                      (daemon)
        sharingd <a95c2cea41b43cc69b0bbe9a03730d45>          448              448   50                      (daemon)
           timed <abc20e52e7ce3cb9832590652d21e7d1>          199              199   50                      (daemon)
          voiced <2823988f691e3715a6e23959590f3c00>          215              215  200                      (daemon)
      calaccessd <77a5672f2f653f64acf7367126a8d173>          219              219  200                      (daemon)
     SpringBoard <3c0e305139b331c6b37d2e9516f5804f>        10476            10476   50                     
      backboardd <d61df126c4673b25bbfe5d9024be1d48>         7615             7615   50                      (daemon)
      aggregated <a5dda46586ba3a3cbb298bd8aa545e50>          454              454   50                      (daemon)
       lockdownd <6f28a28a0025348aa5361078e41914e3>          193              193  100                      (daemon)
    fairplayd.A2 <6cae0c124e1830598a43f6b4d790917f>          132              132  100                      (daemon)
         configd <c57db43e53a73f8a9360f4d0d9001704>          446              446  100                      (daemon)
       fseventsd <5c909a70b62f33c8856e3158834ba071>          242              242  100                      (daemon)
        BTServer <3933a8148924316b9f19dd3d10a23f00>          254              254  100                      (daemon)
       distnoted <38616bd8864034e7bc741f8bd7313349>          117              117  100                      (daemon)
  UserEventAgent <a3c7e56924ec3690a994a75a0ea79ee8>          502              502   50                      (daemon)
        networkd <84dfdb49c24132fa8dd10520deb16645>          423              423  100                      (daemon)
     eapolclient <534b8549294d384c9bec206a272a29cd>          141              141  100                      (daemon)
notification_pro <852af3fe832e3cc3a3f31d05511a5482>          114              114  100                      (daemon)
             ubd <ca1400009326346db54f834cc8b740d2>          491              491  100                      (daemon)
          cplogd <148e9e2ff86130ecb63423c183f68da5>          130              130  200                      (daemon)
     touchsetupd <02780826b4263a7498bda167721b5f8c>          153              153  200                      (daemon)
     debugserver <4f8d671c17be36d2bb9d3b30da9dd947>          164              167  200                      (daemon)
      DTMobileIS <cd97740bdfd833b69d543b3044065d20>          186              186  200                      (daemon)
CommCenterClassi <b836b786e0cb3785a18a94e9b13c9991>          314              314   50                      (daemon)
         notifyd <35afacabfed73771889e72a017479709>          183              183  100                      (daemon)

**End**

I have also created a test project where I can reproduce the issue:

https://github.com/kerrmarin/PINRemoteTest

Is there any way to prevent this crash while maintaining performance? Initially, we thought about setting PINRemoteImageManagerDownloadOptionsSkipDecode as one of the options for the manager, but that limits the scroll performance.

Thanks

[rcancro]

It looks like your collection view cells aren't actually being reused. Try adding an empty dealloc method to PINTestCollectionViewCell and putting a break point in there. It never hits.

If the cells aren't being dealloced, then the images aren't either. Eventually this will lead to OOM exception.

[kerrmarin]

I don't think that's the issue. dealloc isn't called because the cells are reused, and therefore not dealloced. Presumably if this controller was popped, then dealloc would be called (as part of the collection view being dealloced).

[garrettmoon]

What type of iPad are you testing on? Does it happen with all devices, just at different times?

[kerrmarin]

Hi @garrettmoon I've tested on an iPad Mini (non retina) on iOS 7 (the one that produced the crash log above) and on an iPad (retina) on iOS 8. It happens on both devices but I don't have a specific time at which this happens, simply after scrolling up and down for about 30 seconds on the mini and about a minute on the iPad (relatively rapidly).

[garrettmoon]

I'll take a look and see if I can't reproduce it on my local test device and investigate :)

[garrettmoon]

Ok, so here's the issue, we're not receiving memory warnings. Why? This stack overflow has some explanations:

http://stackoverflow.com/a/10009151/262648

It may be that we should set a default memory limit on the PINMemoryCache instance that PINRemoteImage uses and I'd be happy to accept a pull request that had some considered reasoning along with it.

In the meantime, you should be able to prevent this issue by setting those limits yourself. Adding this to the application did finish launching method prevented the issue in your test case:

//Set the limit to store 100 images that are 1000 x 1000 pixels
[[[[PINRemoteImageManager sharedImageManager] cache] memoryCache] setCostLimit:(100 * 1000 * 1000)];

[garrettmoon]

Feel free to open this back up if you think there may be another issue going on here.

[garrettmoon]

The calculation of cost for images stored into the cache could also be improved :)

[kerrmarin]

Thanks for the quick replies. I'll have a look tomorrow morning.

Great stuff 👍

[kerrmarin]

Hi @garrettmoon -- I've made the changes (even set the cost limit to 1 image of 1000 * 1000) and I can still reliably reproduce the crash. I'll investigate further and report back. I'm unable to re-open this issue.

[garrettmoon]

I don't have a low end device to test on, however I'm betting that this has to do with the size of the images being so large and trying to display them all at once. By the way, you'll get huge performance wins if you download images that are the correct size, both in data and scrolling perf.

[garrettmoon]

Re-opened for you.

[kerrmarin]

Thanks Garrett. It doesn't necessarily have to be a low-end device. I've managed to get it to crash on an iPhone 6 also. I don't think it has to do with the amount of memory used, but with the speed at which the memory usage increases (e.g. many images decoding at the same time?) which would explain why limiting the memory cache to just one image doesn't cut it either. Yes, having the right image sizes would be the dream! Unfortunately its not yet possible :(.

I've not had a chance to look at this properly, but I hope to put some time into it over the weekend. Thanks for your help so far.

[garrettmoon]

Sure! The speed at which it's decoding the images definitely could be an issue. Here's something to try:

- (void)setMaxNumberOfConcurrentOperations:(NSInteger)maxNumberOfConcurrentOperations
                                completion:(dispatch_block_t)completion;

Change that to a small number and see if that addresses the issue (possibly in conjunction with the change in cache size). It's set to NSOperationQueueDefaultMaxConcurrentOperationCount by default.

[garrettmoon]

Hey @kerrmarin I'm going to close this issue, let me know if you feel it should be reopened.