Skip to content
X-ray project - information controller transparency tools.
JavaScript Jupyter Notebook TypeScript R CSS HTML Other
Branch: xray-refine
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
analysis add function to blank out repeated rows in table Apr 9, 2018
companydata added host2co changes Jul 22, 2016
data-controller-indicators Merge branch 'master' of https://github.com/sociam/xray into xray-refine Aug 29, 2017
dci-2.0-support added crunchbase display Aug 21, 2017
dci-2.0 exprss changes by emax Oct 26, 2017
mitm nodata Jul 7, 2017
mitm_out merge conflict of company_details.json Oct 26, 2017
monkey scripts for instrumenting monkey Jul 6, 2017
science exprss changes by emax Nov 2, 2017
scraping for downloading apks Jul 6, 2017
static removed conflicting files Aug 29, 2017
.bowerrc
.gitignore added checkpoints Oct 26, 2017
.jshintrc experimental framework Aug 21, 2016
README.md
bower.json experimenter interface Aug 21, 2016
infocon_finder.js added print permissions list to terminal May 9, 2016

README.md

xray / data controller indicators for smartphone apps

Pipeline 1: Dynamic analysis

In this pipeline we will donwload and install apps from the app store, set up mitmproxy, interact with the apps, and do some analysis on their traffic.

capture logs (dynamic approach)

  1. check out the repo
  2. create a virtualenv, enter the virtualenv; pip install mitmproxy
  3. create your own mitm-config.json based on mitm-config.sample.json to customise your environment
  4. run the script like mitmdump -s mitm-save.py -p 8081

While you were interacting with an app on your device you should see traffic on your local console, and logs to be saved in the directory specified in your mitm-config.json file.

process the logs

  1. create a copy of config.sample.json called config.json
  2. you need bower, make sure you have it (or npm install -g bower)
  3. at the top level, run bower install
  4. go to the mitm directory
  5. run npm install
  6. make a 'mitm_out' directory (or whatever destination you specified in config.json)
  7. run node parse-output.js to generate the data files
  8. stand at the base directory, run http-server
  9. point a browser to to localhost:8080/prototypes/p1.html

Automated MITM traffic logging

Another way is to use monkey to simulate user events. Logs traffic data using mitmproxy.

Requires Android Developer Studio tools (for monkeyrunner)

Requires rooted phone with adb root shell access, network log tool with superuser permissions.

Requires working mitmproxy setting up, as described above.

  1. Edit logger.sh to specify packages to log
  2. turn on network logging on the device
  3. Activate the python virtual environment in xray directory, run mitmdump -s mitm-save.py -p 8080
  4. Run logger.sh to start UI fuzzing on packages and logging mitmproxy traffic in the background
  5. When the fuzzer has done its business, Ctrl+C the mitmproxy logging.
  6. There will be a big CSV in the data directory. It can be split into smaller csvs for each app using csplit filename.csv 'endsession_apppackagename' (an ugly solution, will do for now!). Place in the data directory
  7. export network log data from device, place in the data directory
  8. auto-cleaner.py is work in progress, it uses the network log to filter out non-app traffic from the mitmproxy log csv file.

Pipeline 2: Static analysis

In this pipeline we will install apps from the app store, using an automated UI on an android device, pull the APKs from the device, do some static code analysis on the APKs using LibRadar, and store the results of further analysis.

Dependencies

Ensure you have JDK version 1.8.0_25

Downloading APKs via the app store

Requires root access to the android device.

  1. Install android-developer-tools.
  2. Get the device id adb devices
  3. Modify install-from-store-random-motog.py with the device id.
  4. Start ADB demon as root adb root
  5. Run python install-from-store-random-motog.py

This should result in the APK files being written to a directory outside the repo called apks

Static analysis

  1. Adjust the path in static/config.json to point to your APK folder
  2. in the static directory, run node ./trie.js
  3. this should write the LibRadar output to json files in the output directory
  4. Todo: hostsearch.py
You can’t perform that action at this time.