Sample program that crashes the new iOS VM, Part 2

Original issue 1339 created by codenameone on 2015-02-06T23:21:17.000Z:

This is a continuation of issue 1304. The old test program submitted with 1304 served its purpose and now runs after the gc changes made. However my real app still has problems. This is a new test app to reproduce the problem.

In order to run this app you will need to set the property ios.fileSharingEnabled to true and you will need to copy the sample data file to the app on your device using iTunes.

The attached program causes the new iOS VM to run out of memory and crash. Run the program, select Start and then Import. The first time you do this it should work. When it finishes, repeat the above steps and it should crash.

The key to getting this process to crash was moving to a second form. I had the same ImportThread in the main form and it would run fine. But after moving it to a second form it started to crash. My real app uses a second form.

I have run this same app 10 times in a row on the old VM without a problem.

Comment #1 originally posted by codenameone on 2015-02-08T13:44:56.000Z:

On my device (iPhone 6) it doesn't crash after many repeated tries. I do see it taking up more RAM.
We made some improvements related to memory usage in a separate issue and I also made a change to make the GC even more aggressive which might resolve this issue finally. As to performance, I've made a few small improvements which could be significant. You create many strings in reading and this is an expensive process mostly the UTF-8 encoding process. When you construct the CSVParser you do so without specifying encoding which in the new VM defaults to UTF-8 and can reach a relatively slow execution path. If you would specify US-ASCII as the encoding this should work well on the old/new VM and would be much faster assuming the data is actually US-ASCII and not UTF-8.

Comment #2 originally posted by codenameone on 2015-02-08T19:24:53.000Z:

I rebuilt my app and it is better, but it is still crashing.

The results do vary be device but it has been fairly consistent for each device. I am using the row count displayed in the app as a way to measure this.

You do need to run this on an actual device. I had tried building the source code with Xcode and running on my Mac in the simulator and it always worked.

To test I kill any existing app and start from scratch. The first screen contains the Start button, I hit that then import. I keep the screen alive during this test. When the import completes I touch Done and am back on the Start screen. I repeat the process by touching Start again. I am not stopping the app, it is multiple iterations with the same app instance.

When I reported this problem I was getting through one iteration, when I hit Start again it would run to row 700 then crash. This is on an iPod 5G with the latest version of iOS 8. Now on the same device I am getting up to row 1200 of the second iteration before it crashes.

The results on an iPad Mini were a little different. Before I got thru 2 iterations and it crashed on the 3rd one at row 500. Now it gets thru 3 full iterations and crashes on the 4th one at row 400.

I don't have an iPhone to test with. If on your device it gets through multiple iterations without crashing yet the memory usage is going up, to me this indicates a problem. Each time you hit the Start button it is starting the import on a new thread, the import completes and the thread terminates. So I think the memory usage should be consistent after each iteration completes.

I an using UTF-8 for the data. What I am doing is copying a database. I had reported an issue with accented characters when using the Database class, issue 1317, which you fixed. But to get those accented characters in I need to use something more than US ASCII right? My app is also used in many different countries. This is always something I have struggled with but I think I have to use UTF-8.

I am not overly concerned about performance.

Comment #3 originally posted by codenameone on 2015-02-10T09:47:40.000Z:

I don't have an iPod 5g but I do have a 4g one (with OS 6.1) running on that device it crashed around 200. I just made a set of fixes that brought it all the way up to 600 before crashing.
The chief bug I found was that the optimizer optimized away the finalizer methods since it assumed they weren't called so this needed a special case to keep them in place. This effectively created a situation where every String allocation that created an NSString equivalent in the native (e.g. when doing the query to sql) would effectively leak.

Comment #4 originally posted by codenameone on 2015-02-12T07:51:39.000Z:

After a long series of tune ups and improvements to the GC I just ran two cycles of the test case on my iPod 4g so it seems the issue is now fixed. The performance for this specific test case is still slower than the old VM so I'll keep that around and try to improve that as well.

Comment #5 originally posted by codenameone on 2015-02-12T21:11:36.000Z:

I rebuilt my test app and am now able to get through 5 full iterations on both my iPod and iPad, that is real progress.

I stopped at 5 figuring that is enough, for all I know it could go on forever :-)

There is one thing I noticed. Even though the test app ran fine, there was a crash log on my device:

Exception Type: EXC_RESOURCE
Exception Subtype: WAKEUPS
Exception Message: (Limit 150/sec) Observed 942/sec over 300 secs
Triggered by Thread: 5

I think this is the problem that Steve mentioned earlier. I don't know what it means, to the user everything is working but this log was there on the device. I also see the wakeup crash log from my real app too.

Unfortunately back in the real world, in my real app, we have taken a step backwards. After the import completes and the user hits Done, I reconnect to the database and display it in a list. This was working fine, if the import completed, I could display the database. Now it crashes every time on both my iPod and iPad. I hit Done, briefly see a screen full of data and then I am back on my home screen.

I realize this could be something totally different and if so I can come up with a new test case. This can take a while so I just wanted to run this by you first to see if anything jumps out at you. The crash log is attached, this time it is an access exception:

Exception Type: EXC_BAD_ACCESS (SIGSEGV)
Exception Subtype: KERN_INVALID_ADDRESS at 0x61917cf6
Triggered by Thread: 0

If you don't see anything, I will try and create a test case.

Comment #6 originally posted by codenameone on 2015-02-13T04:51:30.000Z:

This seems like something totally different. I assume you created the test around 7 hours ago close to when you filed the issue. We had a major regression with one of the fixes which should have been live about 9 hours ago. Assuming this still happens we'll need a test case again.

Comment #7 originally posted by codenameone on 2015-02-13T18:41:53.000Z:

Still crashing. Something changed, this was working, now it is crashing every time.

I will try and come up with another test program and open a new issue.

Comment #8 originally posted by codenameone on 2015-02-14T06:30:54.000Z:

I built a new version and ran two cycles in a row on my iPod 4g without a problem.

Comment #9 originally posted by codenameone on 2015-02-14T13:28:40.000Z:

I can still run multiple iterations here too, that problem does seem top be fixed. I'm sorry if that wasn't clear. The problem now is that in my real app I go on to use the database and that is crashing. This is new. It seems to be related to opening and closing databases. That seems to cause GC to crash freeing objects. I have created a new issue on this, issue 1348.