Android Compilation Process and Binaries (APK, DEX, OAT, ODEX, VDEX, ART)

[connglli]

Compilation

• Compiler
  • dex
  • d8
• Optimizer (minimization, shrinking, obfuscation, …)
  • ProGuard & DexGuard
  • R8
• Runtime
  • Dalvik Virtual Machine (DVM – JIT, Just-In-Time)
  • Android Runtime (ART – AOT, Ahead-Of-Time)

References

• Android Formats

ART and DVM

At installation time, the installer will separate the .dex, native code and resources.
At the first time of running,

• DVM will use dexopt to optimize the .dex file to a .odex file, e.g. using some efficient bytecode (opcode) (JIT)
• ART will compile the .dex file into native code (.oat or .odex), i.e., the ELF file, and directly run it (AOT).

References

• https://stackoverflow.com/questions/26254538/difference-between-dexopt-and-dex2oat
• https://source.android.com/devices/tech/dalvik

APK

On one hand, an .apk file is no more than a general zip, i.e., generally, as long as one wants to, one can put any type of file to it

On the other hand, an .apk file is a specific zip, where, the AndroidManifest.xml describes the semantic of the it, and semantically, the description of AndroidManifest.xml and files in the .apk satisfy the constraint of Android platform; generally, there are three type of file apk

• manifest: the binary AndroidManifest.xml file
• code: the dalvik byte code files
  • classes.dex, classes2.dex, ...
• libraries: dependent libraries
  • lib/x86/: dependent x86 shared objects (ELF .so)
  • lib/x86_64/: dependent x86_64 shared objects (ELF .so)
  • lib/armeabi/: dependent arm32 shared objects (ELF .so)
  • lib/armeabi-v7a/: dependent arm32 (v7a stands for a version) shared objects (ELF .so)
  • lib/arm64/: dependent arm64 shared objects (ELF .so)
  • lib/arm64-v8a/: dependent arm64 (v8a stands for a version) shared objects (ELF .so)
  • lib/mips/: dependent mips shared objects (ELF .so)
• resources: arsc file, and resources used
  • resources.arsc: a binary file illustrating the resources, including the resource table
  • res/: directory saving all resources used (exactly the res dir in developement)
• others: other files, explained by the app and its dependent libraries, e.g.,
  • META-INF/: meta files, the de facto jar meta-inf
  • assets/: assets except resources
  • kotlin/: kotlin related files

An apk is not required to be runnable, it can have resources only like the overlays; actually, almost all things inside Android are packed as an apk, e.g., the Instrumentation test, the overlay

DEX, OAT, ODEX, VDEX, ART

• .dex: DEX bytecode
• .odex: optimized DEX bytecode, also .dex format but optimized (e.g., replacing some instructions to more-efficient instructions) by the deprecated on-device dexopt tool at installation time to get better interpreter performance
  • on newer versions, this file is an .oat ELF file but suffixed with .odex
• .vdex: verified verbose DEX bytecode, also .dex format but with some more meta data to speed up verification; it contains a copy of the original .dex file
• .oat: an ELF file that is generated by dex2oat, this file does not guarantee to contain AOT-compiled code
  • on some versions, .odex format generated by dex2oat may be an alternative to .oat format
  • in older versions, .oat (or .odex) contains a copy of the original .dex file; however, in newer versions (>= Android 8.0), the copy is moved to .vdex, and .oat (.odex) contains only the AOT-compiled code
• .art: part of the runtime’s image (simply, a variant of heap dump of mirrored C++ objects) to speed up application startup (by avoiding the creation of art::mirror::Class etc.); it especially stores some well-known classes (e.g., java.lang.*)

References

• Android Formats