Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP

Loading…

Performance Degradation in Managed Object Mapping #1232

Closed
blakewatters opened this Issue · 65 comments

9 participants

Blake Watters Dmitry Shevchenko Mike Finkenzeller vonbergm cboivin jeffblake Marcel Bender Rynaard Burger Greg Combs
Blake Watters

From the mailing list:

I'm having similar performance issues after upgrading from 0.10 to 0.20. After playing with instruments for a bit I narrowed the issue down to this part of RKManagedObjectMappingOperationDataSource in the method mappingOperation:targetObjectForRepresentation:withMapping:inRelationship:

// If we are mapping within a relationship, try to find an existing object without identifying attributes
    if (relationship) {
       id mutableArrayOrSetValueForExistingObjects = RKMutableCollectionValueWithObjectForKeyPath(mappingOperation.destinationObject, relationship.destinationKeyPath);

        NSArray *identificationAttributes = [entityMapping.identificationAttributes valueForKey:@"name"];

        for (NSManagedObject *existingObject in mutableArrayOrSetValueForExistingObjects) {
           if (! identificationAttributes) {
               managedObject = existingObject;
               [mutableArrayOrSetValueForExistingObjects removeObject:managedObject];
               break;
           }

            NSDictionary *identificationAttributeValues = [existingObject dictionaryWithValuesForKeys:identificationAttributes];

            if ([[NSSet setWithArray:[identificationAttributeValues allValues]] isEqualToSet:[NSSet setWithObject:[NSNull null]]]) {
               managedObject = existingObject;
               break;
           }
       }
   }

This method chews up 60% of my running time during a large mapping operation and it appears that the majority of it is spent in the above code. I am not exactly sure what the purpose of the above code is and from testing it appears that neither break statement is ever reached during any of my mapping operations. Commenting out the entire if statement gives me a significant reduction in running time.

Can anyone provide a bit more insight as to the purpose of the above code?

Thanks

Blake Watters blakewatters was assigned
Dmitry Shevchenko

Yeah I would say that mapping operations on iPhone 4 got really noticeable on 0.2, to the point of bad UI lags

Mike Finkenzeller

I am seeing similar issues as I am in the process of moving from 0.10 to 0.20. A large load operation which previously took 3-5 seconds on 0.10 is now taking 20-30 seconds on 0.20.

Blake Watters

@mfinkenzeller Try commenting out the block of code referenced in this ticket and see if it restores your performance. I am going to tackle this and get a fix merged over the next couple of days -- I just need a spare hour to get into Instruments

Mike Finkenzeller

Yeah. That speed it up a little bit. Now it is closer to the time from 0.10. I played with it more this weekend and ended up handling the core data import manually in my case. My use case is probably different than most users. I am trying to bulk import data from a REST endpoint. Talking about 100,000 - 300,000 items.

The CoreData mapping in RestKit is great but when it comes to bulk imports it is a little slow.

I was able to speed it up a ton by reducing the number of CoreData queries. Basing it on https://developer.apple.com/library/mac/#documentation/Cocoa/Conceptual/CoreData/Articles/cdImporting.html
and using a find-update pattern to significantly drop the number of queries. I was going to generalize it in case anyone else might be interested in it.

Blake Watters

@mfinkenzeller Did you try running your bulk import while using the RKInMemoryManagedObjectCache? This cache does a single fetch to build a mapping of all existing NSManagedObjectID to identification attribute pairs, which enables the find or create behavior to avoid hitting the persistent store with fetches. Its performance profile should look nearly identical to a manual implementation, since all the overhead comes from pounding the store with fetch requests and building predicates.

Mike Finkenzeller

I did and it is surely more performant than the fetch cache but the memory usage ends up being significantly higher. The devices (especially iPod Touches) can not handle that many rows/objects being kept in memory

Blake Watters blakewatters referenced this issue from a commit
Blake Watters blakewatters Rework `RKManagedObjectMappingOperationDataSource` to avoid performin…
…g mutable(Array|Set|OrderedSet)ValueForKeyPath: unless there are existing objects and the `identificationAttributes` are `nil`. refs #1232
e62a986
Blake Watters

I have reworked this to try and avoid the performance penalty. Can one of you gents verify the fix on your app?

vonbergm

Thanks Blake. On the road right now, but will look into this in more detail next week. Just ran a quick test by making the changes to RKManagedObjectMappingOperationDataSource. Will try to profile to code when I get a chance and nail down where the time is spent. For me it does not seem like this code snippet makes much of a difference:

Here is what I got on an iPad 3 for the time from when the objects were received by RestKit until RestKit was done with processing:
New Method: 287.9 seconds for 6242 objects
Old Method: 292.7 seconds for 6243 objects
Commented Out: 303.7 seconds for 6348 objects
(some objects changed on the server between the test runs, but not significantly).

I am using the RKInMemoryManagedObjectCache starting with a database on the client with only a handful of objects. All relationships coming from the server are mapped by foreign key. I am on the development branch from maybe a month ago.

Blake Watters

Bumping forward to 0.20.1. Will want to take a step back and do a comprehensive performance audit once 0.20.0 is out the door. Still believe I improved the performance of the original issue.

cboivin

Hi. Another performance issue that I encountered using RestKit v0.20 :
All these tests were done with RKInMemoryManagedObjectCache, development branch, on an iPhone 4S iOS 6.1.1.
I got a response from server containing 1000 times the exact same object (in my case, a Track object).
These tracks have a relationship with a PlaylistTrack object that give the position of each Track in a Playlist.
(So we have Track <--->> PlaylistTrack <<---> Playlist.)
All these objects' mappings have an identification attribute set.
The mapping of the request objects (the 1000 same Tracks) to corresponding NSManagedObjects took approx. 15.5 seconds.
Mapping of the same objects to NSObjects took approx. 7.5 seconds.
Parsing (from JSon to Foundation objects) took, as expected, the same amount of time.
For comparison purpose, 1000 different Tracks took approx. 24 seconds to be mapped by RestKit, while doing it manually took approx. 9 seconds.

I hope these elements can be of some help.

Blake Watters

@cboivin Thanks. I will try to recreate such a setup and benchmark it

jeffblake

I'm getting these problems too - getObjects is nested association with event --> guests --> tickets

Takes long on sets < 1000

Sets over 2.5 thousand, it will load the first time, and then crash every subsequent time (on device, not simulator)

Instruments is reporting over 3 million allocations

Can always get it to load the first time only when starting from a fresh Core data stack (ie delete app, re open)

Dmitry Shevchenko

@blakewatters can I help somehow by saving my instruments profile? I have a pretty big load around 300 objects with nested objects, loading that takes ~ 15 seconds and 80-90% of iPhone 5 CPU. Looks like a lot of time is spent in mapping operations around NSCache

Blake Watters
Owner

Not sure how helpful it will be. The only gating factor on getting this addressed is that I am head down on GateGuru work and have not had the cycles to look at it. Feel free to take a crack at optimizing it, otherwise I'll get to it as soon as I can.

Dmitry Shevchenko

I can dump a JSON payload we're using too, if it could provide a better test case.

Marcel Bender

@mfinkenzeller Would be interesting to see your approach importing such high number of items. For me (even with the changes @blakewatters made) an import of ~15k items (with nine entities, four of them using relationships) takes about ten minutes on an iPad 3.

Blake Watters

I am working on benchmarking the library today and will be pushing performance changes throughout the week.

Blake Watters blakewatters referenced this issue from a commit
Blake Watters blakewatters Eliminate use of `NSCache` within `RKPropertyInspector`. Profiling in…
… instruments reveals that use of `NSCache` is significantly slower than `NSMutableDictionary` for this performance critical code. refs #1232
1e0dfa8
Blake Watters blakewatters referenced this issue from a commit
Blake Watters blakewatters Add an autorelease pool around the hot mapping loop in `RKMapperOpera…
…tion` as significant time was being spent in autorelease during mapping and it appears to have a beneficial effect on performance. refs #1232
2b98d2b
Blake Watters

@jeffblake @dmishe @flashfabrixx please grab the latest development bits and give it a whirl now. I've done some profiling and found some large wins that accelerate overall mapping performance.

Going to take a look at memory real quick as well.

Blake Watters blakewatters referenced this issue from a commit
Blake Watters blakewatters Back `RKEntityByAttributeCache` with an `NSMutableSet` instead of an …
…`NSMutableDictionary` for performance. refs #1232
fda3b14
Dmitry Shevchenko

I think I see an improvement here, shaved off around 2.5 seconds from our usual 9s sync process. Still not as good as RK 0.1, but it's start! :)

Marcel Bender

Performance improved quite well - seems to be a little win over her. But the memory usage seems to be quite heavy now.

Blake Watters

I have another optimization in the works that will be pushed shortly.

Have not attacked memory just yet.

vonbergm

Thanks Blake, I noticed a clear improvement. for just above 6000 objects it took about 120 seconds before the latest update, and about 95seconds after. that's on the simulator.

Blake Watters

Another potential perf improvement is available on the bugfix/1351-ghost-objects-in-merge-notifications branch. This branch is development + some new code that attempts to intelligently avoid merging an NSManagedObjectContextDidSaveNotification if the target context was already saved. Its unclear how beneficial this will be yet and would love to hear feedback if it helps you guys any.

Blake Watters

I think that the next improvement on this is going to be the introduction of a modificationAttribute property on RKEntityMapping. This property will let you designate an attribute on your entity that will be checked to determine if it needs to be mapped. If the value from the incoming representation matches the current value of the property, then the mapper will immediately skip forward without evaluating any other attributes.

Dmitry Shevchenko

That sounds helpful, but, do you think there's anything that can be done in terms of raw performance? I tried EasyMappings lib the other day, it's super simple but it maps things pretty fast, same nested payload finishes almost instantly. However, it doesn't do any relation connection like RK does, but I think I tested it with relations turned off and it was still faster. Again, if you need any profiles/traces/payloads I would be happy to help.

vonbergm

I don't think the modificationAttribute would make a difference in my use case, the sync logic server only sends down objects that were actually modified since the last time the client loaded them. The case that's most troublesome for me is when the client database is empty and it pulls in 6000+ objects in one GET request. On a device that takes RestKit between 5 to 10 minutes, depending on what device is used.

Blake Watters

@dmishe Can you link me to the library you were using, I'd like to take a look at what they are actually doing to get a better sense for what an accurate comparison is. The reality at the moment is that all of the things that RestKit supports (i.e. dynamic mapping, type transformations, date parsing, object identification, key-value validation, etc) all come at a cost. We can certainly speed individual components up by looking for optimizations and finding ways to disable code paths for unused features. Its just going to take some more cycles focused on performance to get to a really sweet spot.

Blake Watters

@dmishe nevermind, searched CocoaPods and found it @ https://github.com/lucasmedeirosleite/EasyMapping

Taking a look now and will follow-up once I've read through the code.

Dmitry Shevchenko

Disabling features could be a good deal, like, maybe I would tell RK more about my data upfront instead of using introspection. Because my data doesn't change often, but the speed could be crucial.

Blake Watters

Okay @dmishe, looking through here a few things stand out:

  1. It doesn't support any type transformations out of the box aside from dates. You have to implement it yourself using a block
  2. The object identification support is primitive using a fetch request issued for each object. Were you doing your testing against Core Data backed objects or plain old NSObject classes? The RK managed object caches should already wildly outperform this implementation for large data sets, though overhead in other places may close the gap.
  3. There's no support for modifying relationships using mutable (set|array) valueForKeyPath:. This winds up being a really big deal if you are trying to use KVO or NSFetchedResultsController objects. This may be a place that a Boolean flag on the mapping may be a win.
  4. EasyMapping does not do any comparison on attribute values before assigning them. This means that you will get key-value change notifications for all keys in the representation even if they have not changed value. This is also going to result in more dirty managed objects in each save operation.

I'll take a pass through my benchmarking setup again tomorrow and try disabling some of these potentially expensive subsystems and see what kind of speed results from tying each one off.

Dmitry Shevchenko
  1. that bugfix branch only made things slower :(
  2. Excellent points on EasyMappings, thanks for your hard work.
Stephen Craton scraton referenced this issue from a commit
Blake Watters blakewatters Eliminate use of `NSCache` within `RKPropertyInspector`. Profiling in…
… instruments reveals that use of `NSCache` is significantly slower than `NSMutableDictionary` for this performance critical code. refs #1232
e3a66c1
Stephen Craton scraton referenced this issue from a commit
Blake Watters blakewatters Add an autorelease pool around the hot mapping loop in `RKMapperOpera…
…tion` as significant time was being spent in autorelease during mapping and it appears to have a beneficial effect on performance. refs #1232
cfb4f7b
Stephen Craton scraton referenced this issue from a commit
Blake Watters blakewatters Back `RKEntityByAttributeCache` with an `NSMutableSet` instead of an …
…`NSMutableDictionary` for performance. refs #1232
f728d95
Rynaard Burger

@vonbergm is describing the exact same issue I'm facing. I need to load an entity (7 attributes) with just over 12,000 records on my initial sync. My testing / benchmarking was done with the following devices:

iPod Touch 5th Gen: Giving up after 20+ mins
iPhone 4: Same as iPod Touch above
iPhone 5: Between 10 - 15 mins

Blake Watters

@dmishe would you mind answering my question about managed vs. unmanaged objects in your case? Any other notable attributes about the data set would be helpful as well, number of entities, if relationship connections are in use, dynamic mappings, etc.

Dmitry Shevchenko

Sure, I was using managed objects, basically I replaced RK in my sync operation with easymappings call + afnetworking op.

As for the data structure, this is one object from the heaviest API endpoint I have
https://gist.github.com/dmishe/5452527
we have around 300 per user right now, but expect it to grow.

Top level is one MO, nested item is another MO, they're related by id. Item has another relation by it's item_type key. Field image is a transformable type, but I tried to remove it — didn't notice any change in sync time.

Based on my profiles, I don't think relations have any impact either, seems like nesting is to blame mostly, and filling MOs from data.

It's not a pretty API design honestly and a bit dated.

Blake Watters

I am pulling together an RKBenchmark application that can provide a common context for analyzing performance. Will have that pushed up shortly.

Rynaard Burger

Hi Blake, thanks that will be really useful for benchmarking performance. Just a quick question though, it seems not all of use experience performance issue in the same areas. From the comments it seems some of the guys hit performance bottlenecks during mapping while some of us (including myself) experience issue around the volume of data we're pulling from the server. So my question is, I see there are already a few issues that was addressed but which areas of performance do the cover, if related at all?

Blake Watters

@rburger could you elaborate on the problems you are having around volume of data? Is it that a certain amount of information exhausts memory, etc. Right now I am focused on identifying and optimizing performance in the mapping and Core Data layers, but more generally I am spending a few days on performance as a whole. Would love to understand your issues more clearly.

Blake Watters

@dmishe @vonbergm and @rburger Are you guys using the RKInMemoryManagedObjectCache?

Rynaard Burger

@blakewatters No, I'm currently not using RKInMemoryManagedObjectCache and is hitting the Core Data persistent store (SQLite) directly. The performance issues I experience is on a single Core Data entity with 7 attributes (no relationships) I need to sync from my server on a single GET using the sharedManager's getObjectsAtPath. The response comes back in around a minute with just a little over 12,000 JSON objects. At this stage I already have a 200 response from my server but I never get to the "success" part of the block (especially on older devices). So I guess it is having a hard time to persist all 12,000+ entities to the persistent store in one go?

vonbergm

Hi Blake, I am using the in memory cache, below I also included some numbers with the fetch request cache in parenthesis. Some more info on my situation. At times I pull in a large number of objects into an empty database in one GET request. The time it takes to process grows non-linearly with the number of objects. For example, the processing time (excluding network traffic) for 3447 objects is 25.2 (33) seconds, for 6124 objects it climbs to 74.5 (93.8) seconds. On the simulator, device takes much longer. I have on average about the same number of relationships as attributes (excluding the foreign keys) and I don't use nested json but connect the relationships via foreign key.

Dmitry Shevchenko

I tried it, didn't see the difference, just in case I'm using it wrong:

managedObjectStore.managedObjectCache = [[RKInMemoryManagedObjectCache alloc] initWithManagedObjectContext:managedObjectStore.persistentStoreManagedObjectContext];
vonbergm

@blakewatters, when comparing attribute values before writing them (very important feature!), does RestKit distinguish if the object was newly created on the client or updated on the client? There is no need to check on newly created objects. That should give a noticeable performance boost, although it does not address the non-linear growth in processing time I am experiencing. But that might just be unavoidable when mapping relationships.

@rburger, are you also seeing non-linear growth in processing time? When not mapping relationships the mapping time should be almost linear, except for having to search the payload for duplicate primary IDs.

Blake Watters

@rburger If you are trying to import 7k entities without any relationships, try setting the managedObjectCache property to nil on your operation. This will cause it to send messages to nil when it tries to look up objects, which will produce a drastic speed-up in the execution of the operation.

I am about to push up the RKBenchmarking app in a few and post some initial findings.

Blake Watters

@vonbergm That's a great point and an optimization I am going to try adding in a moment.

Blake Watters

@dmishe Something is fishy there… The cache configuration should definitely have a meaningful impact on the execution times

Blake Watters

Benchmark app I am working with is pushed to https://github.com/RestKit/RKBenchmark

Blake Watters

Initial findings from the importer show the following (run on the Simulator):

Benchmark 'Import 5000 Employees with In Memory Store + Fetch Request Cache' took 20.708636 seconds.
Benchmark 'Import 5000 Employees with In Memory Store + In Memory Cache' took 2.149140 seconds.
Benchmark 'Import 5000 Employees with In Memory Store + nil Cache' took 1.957328 seconds.
Benchmark 'Import 5000 Employees with SQLite Store + Fetch Request Cache' took 21.108720 seconds.
Benchmark 'Import 5000 Employees with SQLite Store + In Memory Cache' took 2.071678 seconds.
Benchmark 'Import 5000 Employees with SQLite Store + nil Cache' took 1.803768 seconds.

The mapping configuration here is an import of 5000 records that were generated via the Ruby Forgery gem (see the Rakefile in the project) with no connections and no relationships. I'll add additional benchmarking scenarios soon but for now I am only concerning myself with raw mapping performance into Core Data.

I am looking into the Fetch Request cache now to see if I can do anything to accelerate it as @dmishe's findings with EasyMapping would suggest that RestKit is slower somehow. EasyMapping just executes a fetch request for every object, so the performance should be closer.

Rynaard Burger

@blakewatters Thanks Blake, I did set the managedObjectCache to nil on my managedObjectStore as you suggested and did find a very noticeable increase in performance.

Here is my findings with managedObjectCache set to nil pulling around 12,000 records from my server in a single GET operation:

iPhone 4: Just over 4 minutes (previously never getting anywhere and giving up after 20 minutes or so)
iPhone 5: 45 seconds! (previously around 5 - 10 minutes)
Simulator: 20 seconds!

Setting the managedObjectCache to nil is spitting out the following in the console:

Performing managed object mapping with a nil managed object cache:
Unable to update existing object instances by identification attributes. Duplicate objects may be created.

I guess that is the tradeoff I have to make but shouldn't be an issue since there is no relational data anyway and should be good for a initial sync only.

Dmitry Shevchenko

Wow, yeah I don't see a difference so big in using in memory cache, there's definitely something strange.

Blake Watters

@rburger Awesome, I am glad that we were able to get you into a better place.

Blake Watters

@dmishe I have thus far been completely unable to identify any way in which EasyMapping could so greatly out-perform RK while still firing fetch requests for every object. I have stripped out all of the logic in the Fetch Request cache and replaced it with a direct port of the logic from their mapper that just builds and executes a fetch request.

Dmitry Shevchenko

@blakewatters are you saying that in your findings easymappings is on par with RK? Must be the problem on my end then. I'm in the middle of tough week, I may have overlooked something in my benchmarks :/

Dmitry Shevchenko

@blakewatters what would be a good place to hook some timers so that I can see how much is spent on local operation not counting network access?

Marcel Bender

FYI Tested my initialization process with managedObjectStore.managedObjectCache = nil and without relationships set as well. Loading ~15k objects in ten entities took just 167 seconds instead of 543 seconds.

Blake Watters

@dmishe If you look at what I've been doing with the RKBenchmark app, it will give you a great way to hook in. I'm basically just using the RKManagedObjectImporter to bootstrap the mapping phase. That way its only benchmarking the JSON deserialization and mapping phase. Makes it trivial to pop it open in Instruments as well.

Blake Watters

@dmishe To be clear: I am saying that I do not see any way possible for EasyMapping to outperform RestKit (assuming any object identification/lookup is happening) since it executes a single fetch request for each object it looks up. In my benchmarks, the time spent executing fetch requests to find existing object by primary key so far eclipses everything else that optimizations at any other level just don't move the needle. That's the dramatic 20 second -> 2 second drop you see in the benchmarks.

Blake Watters blakewatters referenced this issue from a commit
Blake Watters blakewatters Introduce additional experimental performance optimization. refs #1232
* Add `RKMappingOperatationDelegate` method to enable the data source to instruct the operation to skip performing value comparisons when assigning properties
* Add `RKMappingOperatationDelegate` method to enable the data source to instruct the operation to skip property mapping all together
* Add `modificationKey` attribute to enable the property assignment to be skipped when mapping entities that have not changed
e4f61ad
Blake Watters

Okay gents, I've added a new optimization area specific to Core Data. You can now designate a modificationKey (name may change, would love suggestions) on your RKEntityMapping that will be used to determine if any mapping is necessary on a per instance basis. This will not help with initial naked imports (where the in-memory cache or nil cache is more appropriate), but will accelerate secondary syncing requests where only a subset of the payload has changed. I am using updatedAt from my models at the moment.

I am also pushing up changes to the benchmarking app that highlight the performance profile.

Blake Watters

New benchmarking code is up at https://github.com/RestKit/RKBenchmark

I think that the next things that we are going to start looking at on the performance front are stream based JSON parsing and a new in memory cache that intelligently balances memory use

vonbergm

Very cool. In my case the server rarely sends down objects that have not changed, but it does happen. Will implement this the same way, by checking the updatedAt. One quick question. How do you compare the dates? I have had issues in the past because I parse the dates to json using date formatters with millisecond accuracy only. That means that a parsed date coming from the server will be different than the date stored in core data on the order of fractions of a millisecond. This will give a lot of false positives. If you only format to second accuracy the problem gets worse of course.

Greg Combs
Collaborator
Blake Watters

Okay, I have been doing a bunch more work on the performance side of things that isn't yet ready for a push. For transparency, here's what's up:

  1. The in memory entity caches are now asynchronous. This means that when objects are added/removed to them, it returns immediately so mapping can continue. There's now completion blocks if you actually need insight into the status. From an object mapping perspective, this just means that the mapper resumes work more quickly while the caches update in the background.
  2. Relationship connections have been tweaked. Previously we were emitting 1 operation per relationship per object, now we are doing all connections for an object at once.
  3. Relationship connections now yield access to the managed object context. By making use of the asynchronous cache, more of these operations can run in parallel since the fetching is async and the MOC is only locked during the actual assignment of the relationship. Previously they context was locked during the entire main method of the operation.

These changes are much more aggressive than the last round so I expect a few days of life on this branch before coming down to main.

I am planning to get this perf work merged down, then tackle the few highest priority bugs and cut 0.20.1 next week.

Dmitry Shevchenko

Sounds great! :+1:

Blake Watters blakewatters referenced this issue from a commit
Blake Watters blakewatters Rework caches and connections to improve performance. refs #1232
* Eliminate redundant use observers across `RKEntityByAttributeCache` instances. Instead, `RKInMemoryManagedObjectCache` does all observing.
* Adjust caches to work primarilly off of `NSSet` of managed objects instead of single instances
* Rework caches to perform asynchronous cache management activities instead of blocking. Add `completion` blocks for callbacks on async activities.
* Switch `RKInMemoryManagedObjectCache` to use its own private context instead of the persistent store context to avoid potential deadlocks during async cache management
* Rework `RKRelationshipConnectionOperation` to handle an array of `RKConnectionDescription` objects instead of a single one. This reduces the amount of operations minted in proportion to the number of connections used.
5b2e613
Blake Watters blakewatters referenced this issue from a commit
Blake Watters blakewatters Improve search indexing performance by batching objects into a single…
… `NSOperation` instead of minting one per managed object to be indexed. refs #1232
b2a9f0a
Blake Watters
Owner

I merged this work into feature/1232-improve-performance yesterday. I also merged a tweak to the managed object search indexing that vastly accelerates heavy indexing operations.

My benchmarking is showing notable improvements in the speed of everything involving the in memory cache. Total execution time is now very close to a pure nil cache (in which no objects are identified and everything is created).

The modificationKey improvement cuts total mapping time in half for updates.

Blake Watters
Owner

Merged to development.

If anyone is interested, I have rounded up benchmarking results across a few of the interesting SHA's during the lifecycle of this issue: https://docs.google.com/a/restkit.org/spreadsheet/ccc?key=0AmLegXVHvNdpdDJJUWNnMEFjbVoxTGRfUl8xX2NaSUE#gid=0

The first iteration was taken as a baseline from before the optimization work began. Notice that the delta between the In Memory cache and the nil (no cache) converge. The fetch request cache, while still slow, also saw modest gains.

vonbergm

Thanks a bunch Blake!
I pulled the new code yesterday and did a quick test run on my data. Using the in memory cache and pulling in 6227 new objects on an iPad 3 the mapping took 340 seconds with the old code and only 61 seconds after updating. That's a huge improvement!
--jens

Igor Stojanovic stojanovicigi referenced this issue from a commit in CenterDevice/RestKit
Blake Watters blakewatters Rework `RKManagedObjectMappingOperationDataSource` to avoid performin…
…g mutable(Array|Set|OrderedSet)ValueForKeyPath: unless there are existing objects and the `identificationAttributes` are `nil`. refs #1232
1c6a45f
Igor Stojanovic stojanovicigi referenced this issue from a commit in CenterDevice/RestKit
Blake Watters blakewatters Eliminate use of `NSCache` within `RKPropertyInspector`. Profiling in…
… instruments reveals that use of `NSCache` is significantly slower than `NSMutableDictionary` for this performance critical code. refs #1232
21b5fd0
Igor Stojanovic stojanovicigi referenced this issue from a commit in CenterDevice/RestKit
Blake Watters blakewatters Add an autorelease pool around the hot mapping loop in `RKMapperOpera…
…tion` as significant time was being spent in autorelease during mapping and it appears to have a beneficial effect on performance. refs #1232
f6f1dee
Igor Stojanovic stojanovicigi referenced this issue from a commit in CenterDevice/RestKit
Blake Watters blakewatters Back `RKEntityByAttributeCache` with an `NSMutableSet` instead of an …
…`NSMutableDictionary` for performance. refs #1232
61e2eb8
Igor Stojanovic stojanovicigi referenced this issue from a commit in CenterDevice/RestKit
Blake Watters blakewatters Rework the managed object importer so that you can benchmark differen…
…t cache backends. refs #1232
da35a72
Igor Stojanovic stojanovicigi referenced this issue from a commit in CenterDevice/RestKit
Blake Watters blakewatters Introduce additional experimental performance optimization. refs #1232
* Add `RKMappingOperatationDelegate` method to enable the data source to instruct the operation to skip performing value comparisons when assigning properties
* Add `RKMappingOperatationDelegate` method to enable the data source to instruct the operation to skip property mapping all together
* Add `modificationKey` attribute to enable the property assignment to be skipped when mapping entities that have not changed
28daa5d
Igor Stojanovic stojanovicigi referenced this issue from a commit in CenterDevice/RestKit
Blake Watters blakewatters Rework caches and connections to improve performance. refs #1232
* Eliminate redundant use observers across `RKEntityByAttributeCache` instances. Instead, `RKInMemoryManagedObjectCache` does all observing.
* Adjust caches to work primarilly off of `NSSet` of managed objects instead of single instances
* Rework caches to perform asynchronous cache management activities instead of blocking. Add `completion` blocks for callbacks on async activities.
* Switch `RKInMemoryManagedObjectCache` to use its own private context instead of the persistent store context to avoid potential deadlocks during async cache management
* Rework `RKRelationshipConnectionOperation` to handle an array of `RKConnectionDescription` objects instead of a single one. This reduces the amount of operations minted in proportion to the number of connections used.
0dd897b
Igor Stojanovic stojanovicigi referenced this issue from a commit in CenterDevice/RestKit
Blake Watters blakewatters Improve search indexing performance by batching objects into a single…
… `NSOperation` instead of minting one per managed object to be indexed. refs #1232
6b32077
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.