An extensible crawler for downloading Android applications in third-party markets.
Python Shell
Latest commit 77a88c3 Dec 17, 2015 @mssun Update XPATH to
Failed to load latest commit information.
crawler Update XPATH to Dec 17, 2015
downloader Fixing proxy setup Oct 15, 2014
repo update README in repo May 27, 2014
.gitignore updated .gitignore Mar 11, 2014 Update May 20, 2014

Android Apps Crawler


Android Apps Crawler is an extensible crawler for downloading Android applications in the third-party markets. It can crawl the download url addresses of applications and automatically download applications into repository.


  • Python 2.6 or up
  • Scrapy 0.22 or up: (didn't have a full test in lower version.)
  • Works on Linux, Windows, Mac OSX, BSD
  • Currently, downloader cannot work on Windows.
  • For Ubuntu users, "Don't use the python-scrapy package provided by Ubuntu, they are typically too old and slow to catch up with latest Scrapy. Instead, use the official Ubuntu Packages."


  • Set the third-party markets you want to crawl in
  • Set the proxy if you have
  • Start crawler:
./ <market name>
  • Start downloader:
./ <database file path> <output directory>


You can set proxy, user-agen, database name, etc in crawler/android_apps_crawler/ file.

Supported Third-party Markets (market names used in

More Android Markets



  • Windows support for downloader.
  • Crawl apps from shared cloud storage link (e.g,,