Mercury

Version-aware Android analysis toolkit

Mercury is a work in progress Android APK research platform by Elkenson for collecting app versions from public sources and analyzing extracted signals in an interactive web UI, for technical and non-technical users.

The project is intended for legitimate research.

Screenshots

What Mercury does

• Fetches app versions from supported sources (uptodown, apkpure, apkmirror)
• Validates and stores downloaded APK/XAPK/APKM artifacts.
• Extracts various signals from APKs
• Ingests metadata into per-package SQLite DBs under apks/.mercury/
• Builds visual analytics (country world map/domain/host/URL/app-graph/heatmap) in the web UI
• Supports queue-based background analysis for multi-user or long-running sessions

Repository layout

• web/ Dash app, layout, and callbacks
• sources/ clients for repositories such as Uptodown, APKPure, and APKMirror
• analysis/ ingest, parsing, figure-building
• scripts/ maintenance and optional utilities, such as finding favicons to augment visualisations
• bootstrap/ first-run setup for AndroZoo SQLite and popular packages index
• apks/ downloaded app files grouped by source/package (filled during running)
• data/ generated metadata artifacts (filled during running, with helper scripts )

Quick start

1) Install dependencies

python3 -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt

2) Configure environment

Create .env.mercury in repo root:

MERCURY_USERS=researcher:change-me
MERCURY_SECRET_KEY=replace-with-long-random-value

Optional paths and toggles:

• MERCURY_DATA_ROOT (default: apks)
• MERCURY_ANDROZOO_DB (default: data/androzoo.db)
• MERCURY_POPULAR_JSON (default: data/popular_packages.json)
• MERCURY_UI (guided or classic)
• MERCURY_AUTH_ENABLE (default enabled)

3) Bootstrap runtime prerequisites (recommended)

Run lightweight setup first:

python3 -m bootstrap setup

What this does:

• verifies key Python imports
• creates expected runtime directories (data/, apks/, icon/logo folders)
• creates data/domain_metadata.csv if missing
• creates data/popular_packages.json placeholder if DB is not ready
• generates data/v2_supertags.yaml when data/dlc.dat_plain.yml exists

Optional local AndroZoo build (only if local CSV/GZ exists):

python3 -m bootstrap setup --build-androzoo-if-local

Remote first-run bootstrap (downloads missing AndroZoo CSV and V2Fly DLC):

python3 -m bootstrap first-run

Equivalent:

python3 -m bootstrap setup --first-run

4) Bootstrap reference data (full)

python3 -m bootstrap

Equivalent targeted steps:

• python3 -m bootstrap androzoo -o data/androzoo.db
• python3 -m bootstrap popular -d data/androzoo.db -o data/popular_packages.json

5) Run the UI

Local-only:

python3 start_mercury.py run

LAN-accessible:

python3 start_mercury.py run-lan

Default port is 8050.

Typical workflow

1. Select a package from corpus or search online in the UI
2. Fetch versions from a source (including drag & drop your own samples/merge with other sources)
3. Select versions to analyze and run/queue a submission
4. Optionally ingest newly downloaded files into db automatically (db will use a lot less space, allowing samples to be deleted if space is limited).
5. Review domain/host/URL/country/network figures

Maintenance & Utility scripts

• scripts/batch_download.py sampled batch downloader by source
• scripts/batch_download_loop.py periodic rerun wrapper, works with wishlist
• scripts/ingest_apks.py ingest APK metadata into per-package SQLite DBs
• scripts/build_app_icon_db.py app-icon extraction/index build (for UI display)
• scripts/build_domain_logo_db.py domain logo dataset builder (can be featured in certain graph plots)
• scripts/get_and_sync_unmapped_domains.py metadata synchronization helper (maintains a list of seen domains from samples, which can later be used to calculate rarity, etc)

Security and compliance notes

This repository is for legitimate research and engineering analysis. Users are responsible for ensuring their use complies with applicable law and each source site's terms.

• Do not commit binaries to this repository.
• Do not commit secrets (.env.mercury, API keys, credentials).
• Keep scraping volume conservative and avoid abusive request patterns.
• Respect source-site restrictions and robots/ToS where applicable.

If a rights holder or platform operator has a query about this repository, please contact the maintainers so it can be reviewed promptly.

Disclaimer

This project is provided "as is", without warranty of any kind. Use at your own risk. The maintainers are not liable for damages, data loss, service disruption, or legal/compliance issues resulting from use.

Development notes

• run.py and run-lan.py currently run Dash with debug=True; use a production WSGI setup for internet-facing deployment.
• Runtime data directories (apks/, data/) are expected to be local and mutable.
• In progress queue/auth behavior is controlled via MERCURY_* environment variables in web/app.py and web/callbacks.py.

Acknowledgements

Mercury benefits from the broader Android research and open-source tooling ecosystem. Special thanks to:

• AndroZoo
• apkeep
• Androguard
• V2Fly community domain list

Mercury includes optional retrievers for third-party APK sources (including APKMirror, APKPure, and Uptodown). Availability and access are subject to each platform's terms.

License

License is to be confirmed. Until a license is published, all rights are reserved by default.