Skip to content
Server for the Echoprint audio fingerprint system
C Python Shell CMake
Branch: master
Clone or download

Latest commit

Fetching latest commit…
Cannot retrieve the latest commit at this time.

Files

Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
bin first commit Jan 12, 2016
ci Force flake8 to use config file Apr 14, 2016
echoprint_server java bindings - oss notices Feb 12, 2016
java java bindings - oss notices Feb 12, 2016
test java bindings - oss notices Feb 12, 2016
testdata first commit Jan 12, 2016
.flake8 Force flake8 to use config file Apr 14, 2016
.gitattributes Mark other language folders as vendored Apr 14, 2016
.gitignore Add travis-CI Apr 14, 2016
.travis.yml Add travis-CI Apr 14, 2016
CMakeLists.txt first commit Jan 12, 2016
CONTRIBUTING.md java bindings - oss notices Feb 12, 2016
LICENSE java bindings - oss notices Feb 12, 2016
README.md Corrected typo in README.md Aug 29, 2016
echoprint_server_python.c Fixes memory leak in echoprint_server_python wrapper Nov 7, 2016
libechoprintserver.c java bindings - oss notices Feb 12, 2016
libechoprintserver.h java bindings - oss notices Feb 12, 2016
setup.py java bindings - oss notices Feb 12, 2016

README.md

echoprint-server

Build Status License Platforms supported: Linux and OS X

A C library, with a Python extension module and Java bindings, for fast indexing and querying of echoprint data.

Installation

The standalone C library is built with CMake. This step is required for using the Java (but not for the Python) bindings.

To build the Python extension module , run python setup.py install.

Usage

The rest of this file documents the usage of echoprint-server via the Python extension module, through a set of convenience scripts in the bin/ directory.

For the Java bindings, please refer to the UsageExample.java file.

The echoprint code generator, used to convert audio files into echoprint strings, can be found here: echoprint-codegen.

WARNING

The library uses a custom binary format for speed. At this point, ENDIANNESS IS NOT CHECKED so moving index files between machines with different architectures might cause problems. The code has been tested on little endian machines.

The Java code for creating indices explicitly assumes a little endian architecture.

echoprint-decode

Convert a codestring as output by echoprint-codegen into the corresponding list of codes represented as comma-separated integers.

Usage:

echoprint-codegen song.ogg > codegen_output.json
cat codegen_output.json | jq -r '.[0].code' | echoprint-decode > codes.txt

codes.txt will look like:

150555,1035718,621673,794882,40662,955768,96899,166055,...

This script only outputs the echoprint codes, not the offsets. jq is a command line tool to process JSON strings, it can be found here.

echoprint-inverted-index

Takes a series of echoprint strings (one per line) and an output path. Writes a compact index to disk.

Usage:

cat ... | ./echoprint-inverted-index index.bin

index.bin format is binary, see the implementation details below.

If more than 65535 songs are indexed, the output will be split into blocks with the following naming scheme:

index.bin_0000
index.bin_0001
...

Optionally the -i switch switches the input format to a comma-separated list of integer codes (one song per line).

echoprint-inverted-query

Takes a series of echoprint strings (one per line) and a list of index blocks. For each query outputs results on stdout as json-encoded objects.

Usage:

cat ... | ./echoprint-inverted-query index-file-1 [index-file-2 ...]

where the input is an echoprint string per line;

Each output line looks like the following:

{
  "results": [
    {
      "index": 0,
      "score": 0.69340412080287933,
    },
    {
      "index": 8,
      "score": 0.56301175890117883,
    },
    {
      "index": 120,
      "score": 0.31826272477954626,
    },
    ...

The index field represents the position of the matched song in the index.

Optionally the -i switch switches the input format to a comma-separated list of integer codes (one song per line).

REST service

The echoprint-rest-service script listens for POST requests (by default on port 5678), with an echoprint string as echoprint parameter. The test-rest.sh shows how to query using curl.

The request is made to host:query/<METHOD> with <METHOD> one of

  • jaccard
  • set_int
  • set_int_norm_length_first

Usage:

echoprint-rest-service index-file-1 [index-file-2 ...]

The optional --ids-file accepts a path to a text file where each line represents an id for the correspondingly-indexed track in the index. If specified, the returned results will have an id field.

Example: querying from audio

Assuming 0005dad86d4d4c6fb592d42d767e117f.ogg is in the current directory, let's cut it from 00:30 to 4:30 and re-encode it as 128 kbps mp3 (to show that echoprint is robust to alterations in the file):

ffmpeg -i 0005dad86d4d4c6fb592d42d767e117f.ogg \
	-s 30 -t 240 \
	0005dad86d4d4c6fb592d42d767e117f_cut_lowrate.mp3

Run the echoprint codegen, extract the echoprint string:

../echoprint-codegen/echoprint-codegen
    0005dad86d4d4c6fb592d42d767e117f_cut_lowrate.mp3 \
    | jq -r '.[0].code' \
    > 0005dad86d4d4c6fb592d42d767e117f_cut_lowrate.echoprint```

Query the service:

curl -s --data \
    echoprint=`cat 0005dad86d4d4c6fb592d42d767e117f_cut_lowrate.echoprint` \
    <server-path>:5678/query

Results should be similar to

{
  "results": [
    {
      "id": "0005dad86d4d4c6fb592d42d767e117f",
      "index": 0,
      "score": 0.34932565689086914
    },
    {
      "id": "ee59c151d679413a80ac4e49ac92c662",
      "index": 698096,
      "score": 0.033668458461761475
    },
    {
      "id": "026526e6a02648668ff9f410faab15be",
      "index": 312466,
      "score": 0.015930989757180214
    },
    ...
  ]
}

Implementation details

Similarity

The similarity between two echoprints is computed on their bag-of-words representations. This means that the codes' offsets are not considered, nor are the codes' multiplicities.

Inverted index binary format

The inverted index is serialized as several blocks, each being a memory dump of the EchoprintInvertedIndexBlock struct defined in the header file.

License 📝

The project is available under the Apache 2.0 license.

Contributing 📬

Contributions are welcomed, have a look at the CONTRIBUTING.md document for more information.

You can’t perform that action at this time.