LibScout is a light-weight and effective static analysis tool to detect third-party libraries in Android/Java apps. The detection is resilient against common bytecode obfuscation techniques such as identifier renaming or code-based obfuscations such as reflection-based API hiding or control-flow randomization. Further LibScout is capable of pinpointing exact library versions including versions that contain severe bugs or security issues.
LibScout requires the original library SDKs (compiled .jar/.aar files) to extract library profiles that can be used for detection on Android apps. Pre-generated library profiles are hosted at the repository LibScout-Profiles.
Unique detection features:
- Library detection resilient against many kinds of bytecode obfuscation (e.g. obfuscations by ProGuard)
- Capability of pinpointing the exact library version (in some cases to a set of 2-3 candidate versions)
- Capability of handling dead-code elimination, by computing a similarity score against baseline SDKs
- Library API usage analysis upon detection
Besides detecting libraries in apps, LibScout conducts API analyses on the library SDKs to determine API compatibility across versions and adherence to semantic versioning. The resulting data has recently been used to build the Android Studio extension up2dep to help developers keeping their dependencies up-to-date.
Library History Scraper (./scripts)
The scripts directory contains a library-scraper python script to automatically download original library SDKs including complete version histories from Maven Central, JCenter and custom mvn repositories. The original library SDKs can be used to generate profiles and to conduct library API analyses (see modules below). Use the library-profile-generator script to conveniently generate profiles at scale.
The scrapers need to be configured with a json config that includes metadata of the libraries to be fetched (name, repo, groupid, artefactid). The scripts/library-specs directory contains config files to retrieve over 100 libraries from maven central and a config to download Amazon libraries and Android libraries from Google's maven repository (350 libraries, including support, gms, ktx, jetpack, ..).
Detecting (vulnerable) library versions
Ready-to-use library profiles and library meta-data can be found in the repository LibScout-Profiles.
LibScout has builtin functionality to report library versions with the following security vulnerabilities.
The pre-generated profiles for vulnerable versions are tagged with [SECURITY], patches with [SECURITY-FIX].
This information is encoded in the library.xml files that have been used to generate the profiles. We try to update the list/profiles whenever we encounter new security issues. If you can share information, please let us know.
|Airpush||< 8.1||> 8.1||Unsanitized default WebView settings||Link|
|Apache CC||3.2.1 / 4.0||3.2.2 / 4.1||Deserialization vulnerability||Link|
|Dropbox||1.5.4 - 1.6.1||1.6.2||DroppedIn vulnerability||Link|
|3.15||3.16||Account hijacking vulnerability||Link|
|MoPub||< 4.4.0||4.4.0||Unsanitized default WebView settings||Link|
|OkHttp||2.1 - 2.7.4
|Certificate pinning bypass||Link|
|Plexus Archiver||< 3.6.0||3.6.0||Zip Slip vulnerability||Link|
|SuperSonic||< 6.3.5||6.3.5||Unsafe functionality exposure via JS||Link|
|Vungle||< 3.3.0||3.3.0||MitM attack vulnerability||Link|
|ZeroTurnaround||< 1.13||1.13||Zip Slip vulnerability||Link|
On our last scan of free apps on Google Play (05/25/2017), LibScout detected >20k apps containing one of these vulnerable lib versions. These results have been reported to Google's ASI program (still under investigation).
- LibScout requires Java 1.8 or higher and can be build with Gradle.
- Generate a runnable jar with the gradle wrapper gradlew (Linux/MacOS) or gradlew.bat (Windows), by invoking it with the build task, e.g. ./gradlew build
The LibScout.jar is output to the build/libs directory.
- Most LibScout modules require an Android SDK (jar) to distinguish app code from framework code (via the -a switch).
Refer to https://developer.android.com/studio/ for download instructions.
- By default, LibScout logs to stdout. Use the -d switch to redirect output to files. The -m switch disables any text output. Depending on the operation mode (see below), LibScout's results can be written to disk in JSON format or JAVA serialization.
- LibScout repo structure in a nutshell:
|_ gradlew / gradlew.bat (gradle wrappers to generate runnable LibScout.jar) |_ assets | |_ library.xml (Library meta-data template) |_ data | |_ app-version-codes.csv (Google Play app packages with valid version codes) |_ lib | pre-compiled WALA libs, Android axml |_ logging | |_ logback.xml (log4j configuration file) |_ scripts | |_ library-specs (pre-defined library specs) | |_ library-scraper.py (scraper for mvn-central, jcenter, custom mvn) | |_ library-profile-generator.sh (convenience profile generator) |_ src source directory of LibScout (de/infsec/tpl). Includes some open-source, third-party code to parse AXML resources / app manifests etc.
- LibScout supports different use cases implemented as modules (modes of operation). Below a detailed description for each module.
Library Profiling (-o profile)
This module generates unique library fingerprints from original lib SDKs (.jar and .aar files supported). These profiles can subsequently be used for testing whether the respective library
versions are included in apps. Each library file additionally requires a library.xml that contains meta data (e.g. name, version,..). A template can be found in the assets directory.
For your convenience, you can use the library scraper (./scripts) to download full library histories from Maven repositories.
By default, LibScout generates hashtree-based profiles with Package and Class information (omitting methods).
java -jar LibScout.jar -o profile -a android_sdk -x path_to_library_xml path_to_library_file
Library Detection (-o match)
Detects libraries in apps using pre-generated profiles. Optionally, LibScout also conducts an API usage analysis for detected libraries, i.e. which library APIs are used by the app or by other libraries (-u switch).
Analysis results can be written in different formats.
- the JSON format (-j switch), creates subfolders in the specified directory following the app package, i.e. *com.foo* will create *com/foo* subfolders. This is useful when coping with a large number of apps. For detailed information about the information stored, please refer to the JSON output specification.
- the serialization option (-s switch) writes stat files per app to disk that can be used with the database module to create a SQLite database (the DB structure can be found in class de.infsec.tpl.stats.SQLStats
java -jar LibScout.jar -o match -a android_sdk -p path_to_profiles [-u] [-j json_dir] [-s stats_dir] [-m] [-d log_dir] path_to_app(s)
Database Generator (-o db)
Generates a SQLite database from library profiles and serialized app stats:
java -jar LibScout.jar -o db -p path_to_profiles -s path_to_app_stats
Library API analysis (-o lib_api_analysis)
Analyzes changes in the documented (public) API sets of library versions.
The analysis results currently include the following information:
Compliance to Semantic Versioning (SemVer), i.e. whether the change in the version string between consecutive versions (expected SemVer) matches
the changes in the respective public API sets (actual SemVer). Results further include statistics about changes in API sets (additions/removals/modifcations). For removed APIs,
LibScout additionally tries to infer alternative APIs (based on different features).
For the analysis, you have to provide a path to the original library SDKs. LibScout recursively searches for library jars|aars (leaf directories are expected to have at most one jar|aar file and one library.xml file).
For your convenience use the library scraper. Analysis results are written to disk in JSON format (-j switch).
java -jar LibScout.jar -o lib_api_analysis -a android_sdk [-j json_dir] path_to_lib_sdks
For technical details and large-scale evaluation results, please refer to our publications:
Reliable Third-Party Library Detection in Android and its Security Applications
Keep me Updated: An Empirical Study of Third-Party Library Updatability on Android