New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Android install of scala 2.9.0 built binary fails (worked with scala 2.8.1) #4620

Closed
scabug opened this Issue May 19, 2011 · 15 comments

Comments

Projects
None yet
1 participant
@scabug
Copy link

scabug commented May 19, 2011

I realize this is not necessarily a scala bug, but because with worked with scala 2.8.1 I thought I'd report it. I've created a pretty through writeup at the following github testcase:

https://github.com/humcycles/dexfailure

In short, if a scala 2.9.0 built jar contains the following code:

class ObservableTest extends ArrayBuffer[Int]

The eventually generated android adk is uninstallable on the android device.

The full description is as follows:

This gitrepo contains a small testcase that demonstrates a bug in the Android
dex installer and/or scala 2.9. I'm happy to update this report with more
details based on feedback from the community.

For now my workaround is to stay with scala-2.8.1.

  • Summary of bug (when generated with scala-2.9.0):

If a generated apk contains the following scala code as a nested class
(or presumably a top level task) the "adb install
dexfailure-testcase_2.9.0-0.1.apk" will fail.

class ObservableTest extends ArrayBuffer[Int]
  • Expected behavior

Install should not fail.

  • Behavior with scala-2.8.1

It is worth noting that the scala-2.8.1 based APK is substantially smaller than
the scala-2.9.0 version (200k vs 450k)

  • Steps to reproduce

  • Pull down this repo. I've included prebuilt working and failing APKs in
    the target subdirectory if you don't want to build from source.

  • If you do want to build from source: cd into the repo and run
    "sbt reinstall-emulator"

  • Note the error message about INSTALL_FAILED_DEXOPT

This failure seems to occur regardless of how much VMmax or RAM is given to the
device.

  • Details on failure:

  • Host reports:

[error] 1023 KB/s (434216 bytes in 0.414s)
[info] pkg: /data/local/tmp/dexfailure-testcase_2.9.0-0.1.apk
[info] Failure [INSTALL_FAILED_DEXOPT]

  • logcat on device shows:
05-19 08:57:56.920: DEBUG/PackageManager(75): Scanning package com.ridemission.dexfail
05-19 08:57:56.925: INFO/PackageManager(75): Package com.ridemission.dexfail codePath changed from /data/app/com.ridemission.dexfail-1.apk to /data/app/com.ridemission.dexfail-2.apk; Retaining data and using new
05-19 08:57:56.925: INFO/PackageManager(75): Unpacking native libraries for /data/app/com.ridemission.dexfail-2.apk
05-19 08:57:57.025: DEBUG/installd(35): DexInv: --- BEGIN '/data/app/com.ridemission.dexfail-2.apk' ---
05-19 08:58:09.505: ERROR/dalvikvm(473): LinearAlloc exceeded capacity (5242880), last=121180
05-19 08:58:09.505: ERROR/dalvikvm(473): VM aborting
05-19 08:58:09.627: INFO/DEBUG(31): *** *** *** *** *** *** *** *** *** *** *** *** *** *** *** ***
05-19 08:58:09.627: INFO/DEBUG(31): Build fingerprint: 'generic/sdk/generic:2.3.3/GRI34/101070:eng/test-keys'
05-19 08:58:09.627: INFO/DEBUG(31): pid: 473, tid: 473  >>> /system/bin/dexopt <<<
05-19 08:58:09.627: INFO/DEBUG(31): signal 11 (SIGSEGV), code 1 (SEGV_MAPERR), fault addr deadd00d
05-19 08:58:09.627: INFO/DEBUG(31):  r0 fffffec4  r1 deadd00d  r2 00000026  r3 00000000
05-19 08:58:09.627: INFO/DEBUG(31):  r4 800a45c0  r5 004e4a8c  r6 005023ec  r7 0001d95c
05-19 08:58:09.627: INFO/DEBUG(31):  r8 00000000  r9 00000000  10 00000000  fp 00000000
05-19 08:58:09.627: INFO/DEBUG(31):  ip 800a4720  sp bea259a8  lr afd19375  pc 80045a4a  cpsr 20000030
05-19 08:58:09.705: INFO/DEBUG(31):          #00  pc 00045a4a  /system/lib/libdvm.so
05-19 08:58:09.715: INFO/DEBUG(31):          #01  pc 0004dc16  /system/lib/libdvm.so
05-19 08:58:09.715: INFO/DEBUG(31):          #02  pc 00068c88  /system/lib/libdvm.so
05-19 08:58:09.715: INFO/DEBUG(31):          #03  pc 0006945c  /system/lib/libdvm.so
05-19 08:58:09.715: INFO/DEBUG(31):          #04  pc 0006a9ae  /system/lib/libdvm.so
05-19 08:58:09.715: INFO/DEBUG(31):          #05  pc 0006ab7a  /system/lib/libdvm.so
05-19 08:58:09.715: INFO/DEBUG(31):          #06  pc 0005b478  /system/lib/libdvm.so
05-19 08:58:09.725: INFO/DEBUG(31):          #07  pc 0005b6ba  /system/lib/libdvm.so
05-19 08:58:09.725: INFO/DEBUG(31):          #08  pc 0005b948  /system/lib/libdvm.so
05-19 08:58:09.725: INFO/DEBUG(31):          #09  pc 00008c42  /system/bin/dexopt
05-19 08:58:09.725: INFO/DEBUG(31):          #10  pc 00008d3c  /system/bin/dexopt
05-19 08:58:09.725: INFO/DEBUG(31):          #11  pc 00008de0  /system/bin/dexopt
05-19 08:58:09.725: INFO/DEBUG(31):          #12  pc 000091aa  /system/bin/dexopt
05-19 08:58:09.725: INFO/DEBUG(31):          #13  pc 00014db8  /system/lib/libc.so
05-19 08:58:09.725: INFO/DEBUG(31): code around pc:
05-19 08:58:09.725: INFO/DEBUG(31): 80045a28 447a4479 ed0cf7d1 20004c09 ee34f7d1 
05-19 08:58:09.725: INFO/DEBUG(31): 80045a38 447c4808 6bdb5823 d0002b00 49064798 
05-19 08:58:09.725: INFO/DEBUG(31): 80045a48 700a2226 eea0f7d1 000436b7 00045275 
05-19 08:58:09.725: INFO/DEBUG(31): 80045a58 0005eb82 fffffec4 deadd00d b510b40e 
05-19 08:58:09.735: INFO/DEBUG(31): 80045a68 4c0a4b09 447bb083 aa05591b 6b5bca02 
05-19 08:58:09.735: INFO/DEBUG(31): code around lr:
05-19 08:58:09.735: INFO/DEBUG(31): afd19354 b0834a0d 589c447b 26009001 686768a5 
05-19 08:58:09.735: INFO/DEBUG(31): afd19364 220ce008 2b005eab 1c28d003 47889901 
05-19 08:58:09.735: INFO/DEBUG(31): afd19374 35544306 d5f43f01 2c006824 b003d1ee 
05-19 08:58:09.735: INFO/DEBUG(31): afd19384 bdf01c30 000281a8 ffffff88 1c0fb5f0 
05-19 08:58:09.735: INFO/DEBUG(31): afd19394 43551c3d a904b087 1c16ac01 604d9004 
05-19 08:58:09.735: INFO/DEBUG(31): stack:
05-19 08:58:09.735: INFO/DEBUG(31):     bea25968  800a9d9c  
05-19 08:58:09.735: INFO/DEBUG(31):     bea2596c  4003cc38  /dev/ashmem/dalvik-heap (deleted)
05-19 08:58:09.735: INFO/DEBUG(31):     bea25970  afd4270c  /system/lib/libc.so
05-19 08:58:09.735: INFO/DEBUG(31):     bea25974  afd426b8  /system/lib/libc.so
05-19 08:58:09.735: INFO/DEBUG(31):     bea25978  00000000  
05-19 08:58:09.735: INFO/DEBUG(31):     bea2597c  afd19375  /system/lib/libc.so
05-19 08:58:09.735: INFO/DEBUG(31):     bea25980  00000000  
05-19 08:58:09.735: INFO/DEBUG(31):     bea25984  afd183d9  /system/lib/libc.so
05-19 08:58:09.735: INFO/DEBUG(31):     bea25988  000128ac  [heap]
05-19 08:58:09.735: INFO/DEBUG(31):     bea2598c  0005eb82  [heap]
05-19 08:58:09.735: INFO/DEBUG(31):     bea25990  004e4a8c  
05-19 08:58:09.735: INFO/DEBUG(31):     bea25994  005023ec  
05-19 08:58:09.735: INFO/DEBUG(31):     bea25998  0001d95c  [heap]
05-19 08:58:09.735: INFO/DEBUG(31):     bea2599c  afd18437  /system/lib/libc.so
05-19 08:58:09.735: INFO/DEBUG(31):     bea259a0  df002777  
05-19 08:58:09.735: INFO/DEBUG(31):     bea259a4  e3a070ad  
05-19 08:58:09.735: INFO/DEBUG(31): #00 bea259a8  000128a8  [heap]
05-19 08:58:09.735: INFO/DEBUG(31):     bea259ac  8004dc1b  /system/lib/libdvm.so
05-19 08:58:09.735: INFO/DEBUG(31): #01 bea259b0  0001d95c  [heap]
05-19 08:58:09.735: INFO/DEBUG(31):     bea259b4  00000000  
05-19 08:58:09.735: INFO/DEBUG(31):     bea259b8  00000000  
05-19 08:58:09.735: INFO/DEBUG(31):     bea259bc  00014000  [heap]
05-19 08:58:09.735: INFO/DEBUG(31):     bea259c0  000128ac  [heap]
05-19 08:58:09.735: INFO/DEBUG(31):     bea259c4  00012f44  [heap]
05-19 08:58:09.735: INFO/DEBUG(31):     bea259c8  4158eb48  /dev/ashmem/dalvik-LinearAlloc (deleted)
05-19 08:58:09.735: INFO/DEBUG(31):     bea259cc  4003cce0  /dev/ashmem/dalvik-heap (deleted)
05-19 08:58:09.735: INFO/DEBUG(31):     bea259d0  00000000  
05-19 08:58:09.735: INFO/DEBUG(31):     bea259d4  000025e8  
05-19 08:58:09.735: INFO/DEBUG(31):     bea259d8  00000000  
05-19 08:58:09.735: INFO/DEBUG(31):     bea259dc  80068c8d  /system/lib/libdvm.so
05-19 08:58:09.785: WARN/installd(35): DexInv: --- END '/data/app/com.ridemission.dexfail-2.apk' --- status=0x000b, process failed
05-19 08:58:09.785: ERROR/installd(35): dexopt failed on '/data/dalvik-cache/data@app@com.ridemission.dexfail-2.apk@classes.dex' res = 11
05-19 08:58:09.805: WARN/PackageManager(75): Package couldn't be installed in /data/app/com.ridemission.dexfail-2.apk
05-19 08:58:09.805: DEBUG/PackageParser(75): Scanning package: /data/app/com.ridemission.dexfail-1.apk
05-19 08:58:09.825: DEBUG/PackageManager(75): Scanning package com.ridemission.dexfail
05-19 08:58:09.835: INFO/PackageManager(75): Unpacking native libraries for /data/app/com.ridemission.dexfail-1.apk
05-19 08:58:09.875: INFO/ActivityManager(75): Force stopping package com.ridemission.dexfail uid=10035
05-19 08:58:09.885: DEBUG/PackageManager(75):   Activities: com.ridemission.dexfail.MainActivity
@scabug

This comment has been minimized.

Copy link
Author

scabug commented May 19, 2011

Imported From: https://issues.scala-lang.org/browse/SI-4620?orig=1
Reporter: Kevin Hester (kevinh)
Affected Versions: 2.9.0

@scabug

This comment has been minimized.

Copy link
Author

scabug commented Oct 19, 2011

Kevin Hester (kevinh) said:
See the attached email thread for some good ideas on how to possibly address this in the scala library (it will probably be at least a year before Android fixes their end):

https://groups.google.com/group/scala-debate/browse_thread/thread/125dd06446c8c078/461456b4244dc407?lnk=gst&q=android&pli=1#461456b4244dc407

@scabug

This comment has been minimized.

Copy link
Author

scabug commented Jan 4, 2012

Nathan Moos (nathan.moos) said:
You should read this article about using multiple dex files in Android: http://android-developers.blogspot.com/2011/07/custom-class-loading-in-dalvik.html
If there was a way to integrate this into sbt-android-plugin or something else that would be the best, then we can go back to happily programming Scala on Android with 2.9.x! ProGuard wouldn't even be necessary until production under this.

@scabug

This comment has been minimized.

Copy link
Author

scabug commented Jan 9, 2012

@jrudolph said:
@nathan, I don't think this trick would help here. The issue mentioned there seems to be limitation of the binary dex format. The problem discussed in this issue is an issue with the dalvikvm runtime. At runtime all dex files and class loaders share one LinearAlloc space (see https://github.com/android/platform_dalvik/blob/master/vm/LinearAlloc.cpp#L87).

@scabug

This comment has been minimized.

Copy link
Author

scabug commented Jan 9, 2012

@jrudolph said:
The underlying problem is that interface tables in dalvikvm grow more than linear for deep interface hierarchies. I documented this issue here: http://code.google.com/p/android/issues/detail?id=22586

@scabug

This comment has been minimized.

Copy link
Author

scabug commented Jan 9, 2012

@jrudolph said:
I finally found time to do another proper test and it seems that at least the example code from this ticket now works with the latest snapshot (scala-compiler-2.10.0-20120105.025731-245.jar). I assume this comes from a fix in Scala which removes redundant interface declarations in bytecodes.

The fix: scala/scala@7a99c03

I checked the resource consumption with my logging dalvikvm and the test app now seems to use only 4MB of LinearAlloc memory which is under the current limit of 5MB. In newer versions of Android (> 4) this limit has been increased to 8 MB so there should be some buffer space for the next time.

@scabug

This comment has been minimized.

Copy link
Author

scabug commented Jan 9, 2012

@jrudolph said:
And please can someone confirm this for the Scala 2.10.0-SNAPSHOT?

@scabug

This comment has been minimized.

Copy link
Author

scabug commented Jan 9, 2012

Jan Berkel (jberkel) said:
Tried a project which failed to install on 2.9.x, can confirm it installs properly on 2.10.0-SNAPSHOT (2.10.0-20120105.025731-245 to be exact).

@scabug

This comment has been minimized.

Copy link
Author

scabug commented Jan 18, 2012

@jrudolph said:
This should be fixed now in upcoming Scala 2.10.x and Scala 2.9.x by reducing the number of redundantly declared interfaces in Scala generated classes. We seem to get down to about 4MB of LinearAlloc space with those fixes so we have a bit of buffer for now. Recent Android versions have 8MB of LinearAlloc space so those should be even safer.

Independently, the related problem in the dalvikvm was now fixed as well. Once the change has trickled into devices (which will probably need several years) this issue should be dead several times by then :)

@scabug

This comment has been minimized.

Copy link
Author

scabug commented Mar 27, 2012

Seth LaForge (sethml) said:
Is this fixed in 2.9.1-1?

@scabug

This comment has been minimized.

Copy link
Author

scabug commented Mar 30, 2012

@jrudolph said:
No but it should be fixed for the upcoming 2.9.2. Have you tried the 2.9.2-RC2? Or can someone else check this?

@scabug

This comment has been minimized.

Copy link
Author

scabug commented May 4, 2012

Alexey Aksenov (ezhik) said:
I successfully predex Scala 2.9.2 library, upload it to emulator and boot with 2.2 image. "LinearAlloc exceeded" message gone.

library.properties
#Fri, 13 Apr 2012 03:37:12 +0200

version.number=2.9.2
maven.version.number=2.9.2
osgi.version.number=2.9.2.v20120330-163119-949a4804e4
copyright.string=Copyright 2002-2011, LAMP/EPFL

adb log:

D/qemud ( 38): client_registration: registration succeeded for client 2
D/dalvikvm( 56): DexOpt: load 628ms, verify 1156ms, opt 44ms
D/dalvikvm( 33): DexOpt: --- END 'scala-library.jar' (success) ---
D/dalvikvm( 33): DEX prep '/data/framework/scala-library.jar': unzip in 302ms, rewrite 2348ms
D/dalvikvm( 33): DexOpt: --- BEGIN 'scala-collection.jar' (bootstrap=1) ---
D/dalvikvm( 60): DexOpt: load 7393ms, verify 1278ms, opt 34ms
D/dalvikvm( 33): DexOpt: --- END 'scala-collection.jar' (success) ---
D/dalvikvm( 33): DEX prep '/data/framework/scala-collection.jar': unzip in 145ms, rewrite 9181ms
D/dalvikvm( 33): DexOpt: --- BEGIN 'scala-immutable.jar' (bootstrap=1) ---
D/dalvikvm( 61): DexOpt: load 12497ms, verify 967ms, opt 21ms
D/dalvikvm( 33): DexOpt: --- END 'scala-immutable.jar' (success) ---
D/dalvikvm( 33): DEX prep '/data/framework/scala-immutable.jar': unzip in 73ms, rewrite 13753ms
D/dalvikvm( 33): DexOpt: --- BEGIN 'scala-mutable.jar' (bootstrap=1) ---
D/dalvikvm( 66): DexOpt: load 10876ms, verify 1442ms, opt 21ms
D/dalvikvm( 33): DexOpt: --- END 'scala-mutable.jar' (success) ---
D/dalvikvm( 33): DEX prep '/data/framework/scala-mutable.jar': unzip in 63ms, rewrite 12575ms
D/dalvikvm( 33): DexOpt: --- BEGIN 'scala-actors.jar' (bootstrap=1) ---
D/dalvikvm( 67): DexOpt: load 100ms, verify 1303ms, opt 13ms
D/dalvikvm( 33): DexOpt: --- END 'scala-actors.jar' (success) ---
D/dalvikvm( 33): DEX prep '/data/framework/scala-actors.jar': unzip in 31ms, rewrite 1547ms
D/AndroidRuntime( 33): --- registering native functions ---
I/SamplingProfilerIntegration( 33): Profiler is disabled.
I/Zygote ( 33): Preloading classes...

I am sure it is fixed now.

BTW I successfully create Scala Android application (DigiSSHD) and publish it on Google Play. You may find a source code at GitHub. I suspect that is most complex application for Android at the moment (4Mb in summary) based on Scala 2.8.2. Unfortunately the new bug #SI-4626 prevent to build it and test with Scala 2.9 :( But library dex work properly at least.

@scabug

This comment has been minimized.

Copy link
Author

scabug commented Jun 15, 2012

Matthew Willis (appamatto) said (edited on Jun 15, 2012 6:37:02 PM UTC):
This is still happening for me with Scala 2.9.2 using android-plugin when I do device installs.

Are there specific language features that cause dexopt to act up? The link to the Android issue tracker mentions complicated inheritance hierarchies, which I would guess Scala generates for multiple inheritance.

For now I will just start rewriting my code so that it doesn't use these features. I imagine that the Android trickle-down of this bug fix will take a couple of years.

@scabug

This comment has been minimized.

Copy link
Author

scabug commented Aug 9, 2013

@soc said:
I think everything which can be done on our side has been done and shipped in recent releases.
Shall we close this issue?

@scabug

This comment has been minimized.

Copy link
Author

scabug commented Jan 29, 2014

@adriaanm said:
Mixed reports, but at least one success. The other did not provide enough info to diagnose. Please open a new ticket if this is an issue, please also consider using 2.11, which has a smaller standard library.

@scabug scabug closed this Jan 29, 2014

@scabug scabug added the help wanted label Apr 7, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment