Major restructuring to support Android

.buildscript/deploy_snapshot.sh

@@ -0,0 +1,26 @@
+#!/bin/bash
+#
+# Deploy a SNAPSHOT including source and Javadoc to JFrog.
+#
+# Adapted from https://coderwall.com/p/9b_lfq and
+# http://benlimmer.com/2013/12/26/automatically-publish-javadoc-to-gh-pages-with-travis-ci/
+
+SLUG="cmelchior/spanner"
+JDK="oraclejdk8"
+BRANCH="master"
+
+set -e
+
+if [ "$TRAVIS_REPO_SLUG" != "$SLUG" ]; then
+  echo "Skipping snapshot deployment: wrong repository. Expected '$SLUG' but was '$TRAVIS_REPO_SLUG'."
+elif [ "$TRAVIS_JDK_VERSION" != "$JDK" ]; then
+  echo "Skipping snapshot deployment: wrong JDK. Expected '$JDK' but was '$TRAVIS_JDK_VERSION'."
+elif [ "$TRAVIS_PULL_REQUEST" != "false" ]; then
+  echo "Skipping snapshot deployment: was pull request."
+elif [ "$TRAVIS_BRANCH" != "$BRANCH" ]; then
+  echo "Skipping snapshot deployment: wrong branch. Expected '$BRANCH' but was '$TRAVIS_BRANCH'."
+else
+  echo "Deploying snapshot..."
+  ./gradlew artifactoryPublish
+  echo "Snapshot deployed!"
+fi

.gitignore

@@ -4,3 +4,4 @@
 .DS_Store
 /build
 /captures
+*.iml

.travis.yml

@@ -0,0 +1,35 @@
+language: android
+
+android:
+  components:
+  - build-tools-23.0.1
+  - android-23
+  - sys-img-armeabi-v7a-android-23
+  - extra-android-m2repository
+  licenses:
+  - android-sdk-license-5be876d5
+
+jdk:
+- oraclejdk7
+- oraclejdk8
+
+after_success:
+- ".buildscript/deploy_snapshot.sh"
+
+branches:
+  except:
+  - gh-pages
+
+notifications:
+  email: false
+
+sudo: false
+
+cache:
+  directories:
+  - "$HOME/.m2"
+
+env:
+  global:
+  - secure: MC2otmh1gHKEzF8PCoU2Intm7HO0zyEoDtfyEzgRPh9IRzjMn8vqLi4S01y3bm9zk3G3EmEFCE6Cr1BbsX0Pv0Mex6vJMjjZmD6wTC6G+fmd2zIvVxAuWLNxve7boADGSvPRFh5pXl/MUrJ0mfcIarmzq40yFMQBfyfcCH3mKV//RjvuqcKS5J0dc+Qddrh4FkOD4a/y4jWn917RU6PvuNRFghHfDFr6iZYxPIP2Y5pTy0xS/ZLrsKM5rybjXTG9Ob7h8eZe/o046Im1RqOuxm7HEoblHgUFyWAsA7RXAoM9vNPAmANUQIl5n/GZ3lwAnyf6HptG5/f7GbPz4s/o6t7FMnYHMG47NT8K6IehFNzPXICmbEuPkQmJsh4nIh5seUh21aKwqkxT4ZRCu/nPWnD+290lOFIIXaN097XmAJX5L4i7mFmH1ykxafcfmp6VaygUQEv3KMJIWwU8ksdtzhaSLHSU63+3DaLnGuAOrSqPDXbyhLqiCbDWRYo9c+rUK8XenkaL/uCQaGGDwrdAiTtTu+mNAWWG+rY63aNtI7fkWYrTSCBHxWe8fi+H+k4cQcr+fHOZNyxsgZJJL8NNJ+LrRj7/M4/tiLwtjRtIK1Bdzat2KI/FBbOMhPRG+yF/rx7z+x3gDCNeFp0V1PMKA+v/mtqDPBIouuj5YGm/15M=
+  - secure: XlmiVL+Ur5rOmTF9jctoSm2Vq37kBn4MvAtr785aEB5qjDamXppzrPcyScSN1Sdw+8y0pRmEu84xbuDiMIzPQgpIjQF+w0e6vwEUfQ+UNjJXbwxIdDfwa+49Q3PJbLOPRO+pvIn8fYRNwWQeFyodIFVQNMZY39xmvouCrwUYr8ue0vXkUHnSm7XwFis7Dhuab09W9i7Mkag9koD+q8JYgVunbwBIsWcYBdTQCyY1Zpcg2no5aHSSI/7oiX8C7icdK6pluUoZGamCyZUS1qaknkxsj70mw2Q11fP4pKB9I1ck40/93iXY9jYSs2Y6R1gh3FDyBzX2097+AmiGCdJvcJSyFgPWc/FDkBg08EgxVx2pSKyf34Di8mb+x6OjtA9h9LS8JaNE+OKKuUcsLhrAhwid2OhyIPoy6ZsTI+web3/kMVvdcVjBZJEiJ/SuVZlXGJ1vOanC6xZC+m/2C/oQXv0bi0E8oejO1TyMwgp6VCHlVoexLINb02QzXrLcpcOx8AZBYzHnljn3wfD9RXprWg8LPsg8brTBhCsL2mwG5vr43ZkXVq6tboS57tLgeKMl0ZpgGP2pe5sEUd1lgZIqDBla11JZNF4VCgLmQfVxDhSFGcIwMyvAIZzlFn5pfZZHoR5Z9e8f9qIUHP08APVwx5CGPmFVGYRAlvu7eIMh9D4=

README.md

@@ -0,0 +1,201 @@
+Spanner
+=====
+
+Spanner is a micro benchmarking framework designed to run on the Android platform.
+
+It is a fork of the original Caliper project for Java started by Google: code.google.com/p/caliper
+
+# Getting started
+
+
+## Getting Spanner
+
+Stable releases of Spanner is available on [JCenter](https://bintray.com/cmelchior/maven/spanner/view).
+
+```
+dependencies {
+  compile 'dk.ilios:spanner:0.2.0'
+}
+```
+
+The current state of master is available as a SNAPSHOT on [JFrog](http://oss.jfrog.org/oss-snapshot-local/dk/ilios/spanner/).
+
+```
+repositories {
+    maven {
+        url 'http://oss.jfrog.org/artifactory/oss-snapshot-local'
+    }
+}
+
+dependencies {
+  compile 'dk.ilios:spanner:0.2.1-SNAPSHOT'
+}
+```
+
+In order to run Spanner benchmarks using JUnit you also need to add the 
+following dependencies.
+
+```
+androidTestCompile 'com.android.support:support-annotations:23.0.1'
+androidTestCompile 'com.android.support.test:runner:0.4.1'
+androidTestCompile 'com.android.support.test:rules:0.4.1'
+androidTestCompile 'junit:junit:4.12'
+```
+
+In order to upload benchmark results to the website you need the following
+permission in AndroidManifest.xml:
+
+```
+<uses-permission android:name="android.permission.INTERNET" />
+```
+
+
+
+
+## Creating a benchmark
+
+* See an example of a standalone benchmark [here]().
+* See an example of a JUnit benchmark [here]().
+
+## Benchmark results
+
+The output from a benchmark will be posted in 3 places:
+- LogCat
+- Json file in `context.getExternalFilesDir()` 
+- Uploaded to Caliper website (if enabled)
+
+The output from Spanner is compatible with the output from Caliper, so if
+enabled in the configuration, the  results can be uploaded to the Caliper website: 
+https://microbenchmarks.appspot.com/
+
+
+## Differences from Caliper (TODO)
+
+* Able to compare against a baseline
+* Support for running in the JUnit framework
+* Running on the same thread starting the benchmark
+* Removed support for VM parameters 
+
+# Benchmarking
+
+## Why should I benchmark? (TODO)
+
+* https://github.com/google/caliper/wiki/JavaMicrobenchmarks
+* https://github.com/google/caliper/wiki/JavaMicrobenchmarks
+
+
+## Benchmarking with Spanner
+
+Each invocation of Spanner is called a *Run*. Each run consists of 1 benchmark 
+class and one or more methods.
+
+A run has different *axis'*, e.g. method to run and parameters to use.
+
+A *Scenario* is a unique combination of these axis'.
+
+An *Instrument* determines what is measured in any given scenario. Most commonly 
+runtime is measured, but you could also measure memory usage or some arbitrary 
+value.
+
+The combination of a *Scenario* and an *Instrument* is called an *Experiment*. 
+An experiment is thus a full description of what test(s) to run and how to 
+measure it.
+
+Running an experiment is called a *Trial*.
+
+Each trial consists of one or more *Measurements*. 
+
+In an ideal world it should be enough to run one trial with one measurement as 
+it would always produce reliable, reproducible results. However, this is not the 
+case as we are running inside a virtual machine and do not have full 
+control over the operating system. For that reason we normally conduct multiple 
+measurement in each trial in order to smooth our irregularities and gain 
+confidence in our results. 
+
+Each trial will output statistics about the measurements like min, max and mean. 
+
+The output from a Spanner benchmark is the results of all the experiments.
+
+
+## Creating a measurement (TODO)
+
+* Detect number of repetitions needed to exceed threshold
+* Warmup
+
+
+## Benchmarking pitfalls
+
+### JIT / AOT (TODO)
+
+Dalvik uses Just-in-time compilation.
+ART uses Ahead-of-time compilation.
+
+Just-in-time compilers will analyze the code while it runs and optimize it, for 
+this reason it is important to do warmup in these kind of  environments.
+
+Ahead-of-time compilers do not modify the code while it is running, as such no
+warmup should be needed
+running on ART 
+
+* Running tests in a different process
+* Warmup
+* JIT: Code being converted to native code
+* ART will intoduce JIT in the future.
+
+
+### Measuring time (TODO)
+
+* Clock drift (System.nanoTime() / System.currentTimeMillis())
+* Clock granularity, make sure to test it.
+* Make sure that test runs longer than granularity
+* Use appropriate system calls for measuring time.
+
+
+### Benchmark variance (TODO)
+
+* Garbage collector
+* Many layers between Java code and CPU instructions
+* Kernel controls scheduler
+* CPU behaves differently under different loads
+
+* Enable fly-mode
+* Disable as many sensors as possible
+* Remove as many apps as possible
+* Minimize GC
+
+
+### Benchmark overhead (TODO)
+
+* Method calls
+* Iterators
+* Getting a timestamp
+
+
+### Compiler optimizations (TODO)
+
+* Compiler can reorder/remove code.
+* Compile to native code.
+
+### Interpreting results (TODO)
+
+* Be mindful of measured overhead.
+* Results do not say anything about the absolute speed.
+
+
+### Math (TODO)
+
+* What is the confidence interval. 
+* What is variance, how to interpret it.
+
+
+## FAQ
+
+**Why spanner?**
+
+Because a Spanner is a much more useful tool than a Caliper when working with 
+Androids.
+
+
+## Resources
+
+- http://jeremymanson.blogspot.dk/2009/12/allocation-instrumenter-for-java_10.html

REPO.md

@@ -0,0 +1,26 @@
+# Using this repo
+
+* Current version is defined in *library/versiont.txt*
+
+**Build a new SNAPSHOT**
+
+    > ./gradlew spanner:build
+
+**Installing a local SNAPSHOT**
+
+    > ./gradlew spanner:publishToMavenLocal
+
+**Publishing a remote SNAPSHOT**
+
+   > ./gradlew spanner:build spanner:artifactoryPublish
+   
+**Release a new version to Bintray**
+
+1) Will remove SNAPSHOT. Tag the commit and bump to next patch version
+
+   > ./gradlew release 
+
+2) Upload to bintray. TODO Find latest tag and use that as version number
+
+   > ./gradlew bintrayUpload 
+   

build.gradle

@@ -7,15 +7,22 @@ buildscript {
     dependencies {
         classpath 'com.android.tools.build:gradle:1.3.0'
         classpath 'com.neenbedankt.gradle.plugins:android-apt:1.8'
+        classpath 'org.jfrog.buildinfo:build-info-extractor-gradle:3.1.1'
+        classpath 'com.jfrog.bintray.gradle:gradle-bintray-plugin:1.4'
+        classpath 'ch.netzwerg:gradle-release-plugin:1.1.0'
+        classpath 'com.netflix.nebula:gradle-extra-configurations-plugin:3.0.3'
     }
 }
 
 allprojects {
+    group = 'dk.ilios'
+    version = file("${rootDir}/library/version.txt").text.trim();
     repositories {
         jcenter()
+        mavenLocal()
     }
 }
 
 task clean(type: Delete) {
     delete rootProject.buildDir
-}
+}