Tests using OCMock run much slower in iOS 9 SDK #253

Closed
adurdin opened this Issue Oct 1, 2015 · 8 comments

Comments

Projects
None yet
5 participants
@adurdin

adurdin commented Oct 1, 2015

Our 3,114 unit tests are running around 4x slower in Xcode 7 (iOS 9 SDK) than with Xcode 6 (iOS 8 SDK):

  • Xcode 6: 33 seconds.
  • Xcode 7: 127 seconds.

We are using OCMock 3.1.2.

Time profiling the unit test run with iOS 8 SDK:

ios8

And with iOS 9 SDK:

ios9

@erikdoe

This comment has been minimized.

Show comment
Hide comment
@erikdoe

erikdoe Oct 1, 2015

Owner

This does really seem to be caused by a change in the new iOS runtime. I've created two very similar projects, one for iOS and one for OS X, that demonstrate that the OS X version is still fast: https://github.com/erikdoe/ocmock-profiling

It might be possible to create a workaround, maybe caching the dynamic subclasses, but I'm hoping that the issue will get fixed in the runtime. The engineering team at Apple is already aware of the problem: https://twitter.com/joar_at_work/status/649330511177912320

Owner

erikdoe commented Oct 1, 2015

This does really seem to be caused by a change in the new iOS runtime. I've created two very similar projects, one for iOS and one for OS X, that demonstrate that the OS X version is still fast: https://github.com/erikdoe/ocmock-profiling

It might be possible to create a workaround, maybe caching the dynamic subclasses, but I'm hoping that the issue will get fixed in the runtime. The engineering team at Apple is already aware of the problem: https://twitter.com/joar_at_work/status/649330511177912320

@carllindberg

This comment has been minimized.

Show comment
Hide comment
@carllindberg

carllindberg Nov 9, 2015

Contributor

One thing you might try is to add all the methods you know you need before calling objc_registerClassPair(). That may (in theory) avoid all of the cache flushes and locking etc. associated with adding those methods, because the class is not fully in the runtime system yet.

Contributor

carllindberg commented Nov 9, 2015

One thing you might try is to add all the methods you know you need before calling objc_registerClassPair(). That may (in theory) avoid all of the cache flushes and locking etc. associated with adding those methods, because the class is not fully in the runtime system yet.

@erikdoe

This comment has been minimized.

Show comment
Hide comment
@erikdoe

erikdoe Dec 31, 2015

Owner

To be honest, I was hoping that this might get fixed in the runtime. With iOS 9.2 the profile looks a bit different, which makes me think that some work is being done. Unfortunately, the slowness is still there. Maybe this is the time to consider work-arounds.

Owner

erikdoe commented Dec 31, 2015

To be honest, I was hoping that this might get fixed in the runtime. With iOS 9.2 the profile looks a bit different, which makes me think that some work is being done. Unfortunately, the slowness is still there. Maybe this is the time to consider work-arounds.

BohdanOrlov added a commit to BohdanOrlov/ocmock that referenced this issue Feb 17, 2016

BohdanOrlov added a commit to BohdanOrlov/ocmock that referenced this issue Feb 17, 2016

BohdanOrlov added a commit to BohdanOrlov/ocmock that referenced this issue Feb 17, 2016

BohdanOrlov added a commit to BohdanOrlov/ocmock that referenced this issue Feb 17, 2016

@BohdanOrlov

This comment has been minimized.

Show comment
Hide comment
@BohdanOrlov

BohdanOrlov Feb 23, 2016

@erikdoe could you please consider my PR? I would appreciate this very much since OCMock 3 performance prevents me from upgrading from OCMock 2.

@erikdoe could you please consider my PR? I would appreciate this very much since OCMock 3 performance prevents me from upgrading from OCMock 2.

@erikdoe

This comment has been minimized.

Show comment
Hide comment
@erikdoe

erikdoe Feb 23, 2016

Owner

I would very much prefer a PR that implements the workaround I suggested above.

Owner

erikdoe commented Feb 23, 2016

I would very much prefer a PR that implements the workaround I suggested above.

@BohdanOrlov

This comment has been minimized.

Show comment
Hide comment
@BohdanOrlov

BohdanOrlov Feb 28, 2016

@erikdoe please have a look on the workaround I implemented as you suggested #285

@erikdoe please have a look on the workaround I implemented as you suggested #285

@JKuzimski

This comment has been minimized.

Show comment
Hide comment
@JKuzimski

JKuzimski May 18, 2016

@erikdoe it looks like with 9.3 OCMock again run fast

@erikdoe it looks like with 9.3 OCMock again run fast

@erikdoe

This comment has been minimized.

Show comment
Hide comment
@erikdoe

erikdoe Jul 26, 2016

Owner

@JKuzimski Yes, when running the profiling projects I see performance of the iOS version is now very similar to the OS X version. So, I'm going to close this as resolved.

Owner

erikdoe commented Jul 26, 2016

@JKuzimski Yes, when running the profiling projects I see performance of the iOS version is now very similar to the OS X version. So, I'm going to close this as resolved.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment