Release 0.7.0 (#73)

* Skip conversion to cffi char[] unless required

* Libraries shouldn't configure logging

* Version bump to 0.6.3

* Improvements to benchmark (#58)

* Refactor Dice coefficient calculation.

* Temporary fiddling with benchmark code.

* Calculate and report popcount speed from native code implementation.

* Give some values more sensible variable names.

* Remove unused import.

* Add documentation.

* Expand reporting of various measurements.

* Comments.

* Update README.

* Bring test suite up-to-date.

* Address Brian's comments.

* Update tests; also test native code version.

* Print popcount throughput; give some variables better names.

* Update README with throughput data.

* Refactor main C++ function to avoid use "constant" memory and avoid new/delete (#55)

* Refactor main C++ function to avoid use "constant" memory and avoid new/delete.

* Refactor Dice coefficient calculation.

* Temporary fiddling with benchmark code.

* Calculate and report popcount speed from native code implementation.

* Give some values more sensible variable names.

* Remove unused import.

* Add documentation.

* Expand reporting of various measurements.

* Comments.

* Update README.

* Bring test suite up-to-date.

* Refactor main C++ function to avoid use "constant" memory and avoid new/delete.

* Address Brian's comments.

* Update tests; also test native code version.

* Print popcount throughput; give some variables better names.

* Feature build on Travis CI (#61)

Run tests with travis ci

* Fix #include file name.

* Use pytest (#68)

* Update README and requirements.txt files.

* Add missing line in README.

* Use pytest on Jenkins.

* Make Jenkins test commands the same as Travis.

* Generate test output and coverage data properly.

* Move 'checkout scm' command to start of function; remove redundant cleaning code.

Fix #65

* Feature use jenkinslibrary (#70)

* Update jenkinsfile to use jenkins library.

* Reduce the number of OSX build and which node in Jenkinsfile (see #71)

* Arbitrary length Dice coefficients (#63)

* Refactor main C++ function to avoid use "constant" memory and avoid new/delete.

* Implement popcount on (almost) arbitrary length arrays.

* First pass at integrating arbitrary length keys. Slows things down a bit.

* Refactor Dice coefficient calculation.

* Temporary fiddling with benchmark code.

* Calculate and report popcount speed from native code implementation.

* Give some values more sensible variable names.

* Remove unused import.

* Add documentation.

* Expand reporting of various measurements.

* Comments.

* Update README.

* Bring test suite up-to-date.

* Refactor main C++ function to avoid use "constant" memory and avoid new/delete.

* Screw everything up by unrolling with C++ templates, apparently.

* Magical argument that makes the compiler generate the correct (performant) code.

* Address Brian's comments.

* Update tests; also test native code version.

* Print popcount throughput; give some variables better names.

* Make some functions static inline.

* Tidy up some expressions.

* Put some braces in the right place; make fn inline.

* Reinstate comment on origin of popcount assembler.

* Make constant a template parameter.

* Comment.

* Complete version working with multiples of 1024 bits.

* Add -march=native compiler option.

* Implementation of arbitrary length CLKs.

* Fix dumb mistakes in updating array pointer and popcounts.

* Tests for arbitrary length popcounts.

* Update some comments.

* Arbitrary length Dice coefficient.

* Rename function.

* Move native dicecoeff calculation into its own function.

* Add tests for native Dice coefficient calculation.

* Move dicecoeff tests to bloommatcher tests; move common bitarray utilities to their own file.

* Simplify slow path / reduce branches in fast path.

* Adapt entitymatcher to arbitrary length CLK interface.

* Remove unused function.

* Update README.

* Address Brian's comments.

* Exit early if filter is zero.

* Specialise popcount arrays calls on array length.

* Fix performance regression.

* Remove storage class specifiers from explicit template specialisations.

* Update README and requirements.txt files.

* Disable unused function.

* Put stars in their proper place.

* Add documentation.

* Prepare changelog and bump version for release 0.7.0

* Add clkhash as dependency (required for benchmark)
Add travis badge to readme

.travis.yml

@@ -0,0 +1,26 @@
+# After changing this file, check it on:
+#   http://lint.travis-ci.org/
+language: python
+sudo: false
+
+python:
+  - '3.6'
+  - 'nightly'
+  - 'pypy3'
+
+env:
+  - INCLUDE_10K=
+
+matrix:
+  allow_failures:
+    - python: 'nightly'
+    - python: 'pypy3'
+
+install:
+  - travis_retry pip install pytest pytest-cov
+  - travis_retry pip install -r requirements.txt
+  - travis_retry python setup.py sdist bdist_wheel
+  - travis_retry pip install -e .
+
+script:
+  - pytest --cov=anonlink

CHANGELOG.rst

@@ -1,3 +1,23 @@
+0.7.0
+-----
+
+Introduces support for comparing "arbitrary" length cryptographic linkage keys.
+Benchmark is much more comprehensive and more comparable between releases - see the
+readme for an example report.
+
+Other improvements
+~~~~~~~~~~~~~~~~~~
+
+- Internal C/C++ cleanup/refactoring and optimization.
+- Expose the native popcount implementation to Python.
+- Bug fix to avoid configuring a logger.
+- Testing is now with `py.test` and runs on [travis-ci](https://travis-ci.org/n1analytics/anonlink/)
+
+0.6.3
+-----
+
+Small fix to logging setup.
+
 0.6.2 - Changelog init
 ---------------------
 

Jenkinsfile.groovy

@@ -0,0 +1,127 @@
+@Library("N1Pipeline@0.0.5")
+import com.n1analytics.git.GitUtils;
+import com.n1analytics.git.GitCommit;
+import com.n1analytics.n1.docker.N1EngineContainer;
+import com.n1analytics.python.PythonVirtualEnvironment;
+
+def isMaster = env.BRANCH_NAME == 'master'
+def isDevelop = env.BRANCH_NAME == 'develop'
+
+VENV_DIRECTORY = "env"
+
+GIT_CONTEXT = "jenkins"
+
+def configs = [
+    [label: 'GPU 1', pythons: ['python3.4', 'python3.5', 'python3.6'], compilers: ['clang', 'gcc']],
+    //[label: 'osx', pythons: ['python3.5'], compilers: ['clang', 'gcc']]
+    [label: 'McNode', pythons: ['python3.5'], compilers: ['clang']]
+]
+
+def PythonVirtualEnvironment prepareVirtualEnvironment(String pythonVersion, clkhashPackageName, compiler, venv_directory = VENV_DIRECTORY) {
+  PythonVirtualEnvironment venv = new PythonVirtualEnvironment(this, venv_directory, pythonVersion);
+  venv.create();
+  venv.runPipCommand("install --upgrade pip coverage setuptools wheel")
+  venv.runPipCommand("install --quiet --upgrade ${clkhashPackageName}");
+  venv.runPipCommand("install -r requirements.txt");
+  String cc = "CC=" + compiler;
+  withEnv([cc]) {
+    venv.runCommand("setup.py sdist bdist_wheel --universal");
+  }
+  venv.runPipCommand("install -e .");
+  return venv;
+}
+
+def build(python_version, compiler, label, release = false) {
+  GitUtils.checkoutFromSCM(this)
+  Exception testsError = null;
+  try {
+    clkhashPackageName = "clkhash-*-py2.py3-none-any.whl"
+    copyArtifacts(
+        projectName: 'clkhash/master',
+        fingerprint: true,
+        flatten    : true,
+        filter     : 'dist/' + clkhashPackageName
+    )
+
+    PythonVirtualEnvironment venv = prepareVirtualEnvironment(python_version, clkhashPackageName, compiler)
+    try {
+      venv.runChosenCommand("pytest --cov=anonlink --junit-xml=testoutput.xml --cov-report=xml:coverage.xml")
+      if (release) {
+        // This will be the official release
+        archiveArtifacts artifacts: "dist/anonlink-*.whl"
+        archiveArtifacts artifacts: "dist/anonlink-*.tar.gz"
+      }
+    } catch (Exception err) {
+      testsError = err
+    } finally {
+      if (!release) {
+        junit 'testoutput.xml'
+      } else {
+        venv.runChosenCommand("coverage xml --omit=\"*/cpp_code/*\" --omit=\"*build_matcher.py*\"")
+        cobertura coberturaReportFile: 'coverage.xml'
+      }
+      if (testsError != null) {
+        throw testsError
+      }
+    }
+
+  } finally {
+    try {
+      deleteDir()
+    } catch (Exception e) {
+      echo "Error during 'deleteDir':\n" + e.toString()
+    }
+  }
+}
+
+
+def builders = [:]
+for (config in configs) {
+  def label = config["label"]
+  def pythons = config["pythons"]
+  def compilers = config["compilers"]
+
+  for (_py_version in pythons) {
+    for (_compiler in compilers) {
+
+      def py_version = _py_version
+      def compiler = _compiler
+      def combinedName = "${label}-${py_version}-${compiler}"
+
+      builders[combinedName] = {
+        node(label) {
+          stage(combinedName) {
+            build(py_version, compiler, label, false)
+          }
+        }
+      }
+    }
+  }
+}
+
+GitCommit commit;
+node() {
+  commit = GitUtils.checkoutFromSCM(this);
+  commit.setInProgressStatus(GIT_CONTEXT);
+}
+
+try {
+  parallel builders
+} catch (Exception err) {
+  node() {
+    commit.setFailStatus("Build failed", GIT_CONTEXT);
+  }
+  throw err
+}
+
+node('GPU 1') {
+  stage('Release') {
+    try {
+      build('python3.5', 'gcc', 'GPU 1', true)
+      commit.setSuccessStatus(GIT_CONTEXT)
+    } catch (Exception e) {
+      commit.setFailStatus("Release failed", GIT_CONTEXT);
+      throw e;
+    }
+  }
+}