Permalink
Switch branches/tags
Nothing to show
Find file Copy path
Fetching contributors…
Cannot retrieve contributors at this time
333 lines (306 sloc) 14 KB
---
# Copyright 2017 Yahoo Holdings. Licensed under the terms of the Apache 2.0 license. See LICENSE in the project root.
title: "Tensor User Guide"
---
<p>
This user guide explains the common steps needed to work with tensors in Vespa:
<ul>
<li>Setting up tensor fields in search definitions</li>
<li>Feeding tensors to Vespa</li>
<li>Querying Vespa with tensors</li>
<li>Ranking with tensors</li>
<li>Constant tensors</li>
<li>Tensor Java API -
<a href="http://javadoc.io/page/com.yahoo.vespa/vespajlib/latest/com/yahoo/tensor/Tensor.html">javadoc</a></li>
<li>Common use cases</li>
</ul>
Most of the contents discussed here is demonstrated in the
<a href="https://github.com/vespa-engine/sample-apps/tree/master/basic-search-tensor">tensor sample app</a>.
For a quick introduction to tensors, refer to
<a href="tensor-intro.html">working with tensors</a>
and <a href="reference/tensor.html">the tensor reference guide</a>.
See also <a href="tensorflow.html">using TensorFlow models</a> or
<a href="onnx.html">using ONNX models</a> which explains how to
use TensorFlow or ONNX models directly in Vespa.
</p>
<h2 id="tensor-document-fields">Tensor document fields</h2>
<p>
In typical use, a document contains one or more
<a href="https://github.com/vespa-engine/sample-apps/blob/master/basic-search-tensor/src/main/application/searchdefinitions/music.sd#L35-L37"> tensor fields</a> to be used for ranking -
this example sets up a tensor field called <code>tensor_attribute</code>:
<pre>
field tensor_attribute type tensor(x{}) {
indexing: attribute | summary
attribute: tensor(x{})
}
</pre>
A tensor requires a type - <code>x{}</code> means <em>sparse</em> dimension,
where <code>x[]</code> means <em>indexed</em> dimension.
For details on tensor fields, refer to the
<a href="reference/search-definitions-reference.html#type:tensor">tensor field</a> in search definitions.
For details on tensor types, refer to the
<a href="reference/search-definitions-reference.html#tensor-type-spec">tensor type reference</a>.
</p>
<h2 id="feeding-tensors">Feeding tensors</h2>
<p>
There are two options when feeding tensors.
<ul>
<li>Convert some other field, for instance a vector of floats,
to a tensor during document processing using the tensor Java API -
refer to the <a href="http://javadoc.io/page/com.yahoo.vespa/vespajlib/latest/com/yahoo/tensor/Tensor.html">tensor Java API</a>.</li>
<li>Feed tensors using the <a href="reference/document-json-put-format.html#tensor">
tensor JSON format - example:</a>:
<pre>
{
"fields": {
"tensor_attribute": {
"cells": [
{ "address" : { "x" : "0" }, "value": 1.0 },
{ "address" : { "x" : "1" }, "value": 2.0 },
{ "address" : { "x" : "2" }, "value": 3.0 },
{ "address" : { "x" : "3" }, "value": 5.0 }
]
}
}
}
</pre>
</li>
</ul>
Here, the <code>x</code>-dimension is sparse, so the indices can be any textual value.
For dense dimensions, indices must be numeric.
See the <a href="https://github.com/vespa-engine/sample-apps/blob/master/basic-search-tensor/music-data-1.json#L8-L15">
sample app</a>.
</p>
<h2 id="querying-with-tensors">Querying with tensors</h2>
<p>
Tensors can not be used in searching, only for ranking.
The tensor can either be supplied in the query string,
or constructed from some other data or data source.
In the latter case, please refer to <a href="http://javadoc.io/page/com.yahoo.vespa/vespajlib/latest/com/yahoo/tensor/Tensor.html">the tensor Java API</a>
for details on how to construct tensors programmatically.
</p><p>
Set the type (indexed or sparse) of the <code>query(tensor)</code> for it to be used in ranking,
using a <a href="query-profiles.html#query-profile-types">query profile type</a>.
This configures the ranking backend that the rank feature named
<code>query(tensor)</code> has type <code>tensor(x{})</code>,
enabling it to effectively compile expressions that use this feature.
Create a file <code>search/query-profiles/types/root.xml</code> in the application package:
<pre>
&lt;query-profile-type id="root" inherits="native"&gt;
&lt;field name="ranking.features.query(tensor)" type="tensor(x{})" /&gt;
&lt;/query-profile-type&gt;
</pre>
Also, configure the default query profile to use this this (or other query profiles as needed):
<pre>
&lt;query-profile id="default" type="root" /&gt;
</pre>
A tensor can be constructed directly from the tensor <a href="reference/tensor.html">literal form</a>.
The corresponding literal form of the tensor in the feeding section is:
<pre>{% raw %}
{{x:0}: 1.0, {x:1}: 2.0, {x:2}: 3.0, {x:3}: 5.0}
{% endraw %}</pre>
Example query (not url-encoded for readability),
using rank profile <em>dot_product</em> and the tensor in <em>query(tensor)</em>:
<pre>{% raw %}
http://host:port/search/?ranking=dot_product&amp;ranking.features.query(tensor)={{x:0}:1.0,{x:1}:2.0,{x:3}:3.0,{x:4}:5.0}&amp;yql=select * from sources * where sddocname contains "music";
{% endraw %}</pre>
</p><p>
Alternatively, use a custom searcher, and send the tensor in a parameter, like:
<pre>{% raw %}
http://host:port/search/?tensor={{x:0}:1.0,{x:1}:2.0,{x:3}:3.0,{x:4}:5.0}&amp;yql=select * from sources * where sddocname contains "music";
{% endraw %}</pre>
<pre>
public class ExampleTensorSearcher extends Searcher {
@Override
public Result search(Query query, Execution execution) {
Object tensorProperty = query.properties().get("tensor");
if (tensorProperty != null) {
Tensor tensor = Tensor.from(tensorProperty.toString());
query.getRanking().getFeatures().put("query(tensor)", tensor);
query.properties().set(new CompoundName("ranking"), "dot_product");
}
return execution.search(query);
}
}
</pre>
This grabs the value of the <code>tensor</code> query parameter, and constructs a
<code>com.yahoo.tensor.Tensor</code> object directly from the value. It then adds this object
to the query as a rank feature. You can also create the Tensor object programmatically.
Refer to <a href="http://javadoc.io/page/com.yahoo.vespa/vespajlib/latest/com/yahoo/tensor/Tensor.html">the tensor Java API</a> and the
<a href="https://github.com/vespa-engine/sample-apps/blob/master/basic-search-tensor/src/main/java/com/yahoo/example/ExampleTensorSearcher.java">tensor sample app</a>.
</p>
<h2 id="ranking-with-tensors">Ranking with tensors</h2>
<p>
Tensors are used during ranking to modify a document's rank score given the query.
Typical operations are dot products between tensors of order 1 (vectors),
or matrix products between tensors of order 2 (matrices).
Tensors are used in <a href="reference/ranking-expressions.html">rank expressions</a> as rank features.
Two rank features are defined above:
<ul>
<li><code>attribute(tensor_attribute)</code>: the tensor associated with the document</li>
<li><code>query(tensor)</code>: the tensor sent with the query</li>
</ul>
These can be used in rank expressions.
Note that the final rank score of a document must resolve into a single double value -
<a href="https://github.com/vespa-engine/sample-apps/blob/master/basic-search-tensor/src/main/application/searchdefinitions/music.sd#L49-L55">
example</a>:
<pre>
rank-profile dot_product {
first-phase {
expression: sum(query(tensor)*attribute(tensor_attribute))
}
}
</pre>
This takes the product of the query tensor and the document tensor, and sums
all fields thus resolving into a single value which is used as the rank score.
In the case above, the value is <code>39.0</code>.
</p><p>
There are some <a href="reference/ranking-expressions.html#built-in-functions">ranking functions</a>
that are specific for tensors:
<table class="table">
<thead></thead><tbody>
<tr><td>map(tensor, f(x)(...))</td> <td>Returns a new tensor with the lambda function defined in <code>f(x)(...)</code> applied to each cell.</td></tr>
<tr><td style="white-space:nowrap;">reduce(tensor, aggregator, dim1, dim2, ...)</td> <td>Returns a new tensor with the <code>aggregator</code> applied across dimensions dim1, dim2, etc. If no dimensions are specified, reduce over all dimensions.</td></tr>
<tr><td>join(tensor1, tensor2, f(x,y)(...))</td> <td>Returns a new tensor constructed from the <em>natural join</em> between <code>tensor1</code> and <code>tensor2</code>, with the resulting cells having the value as calculated from <code>f(x,y)(...)</code>, where <code>x</code> is the cell value from <code>tensor1</code> and <code>y</code> from <code>tensor2</code>.</td></tr>
</tbody>
</table>
These primitives allow for a great deal of flexibility when combined. The above rank expression is equivalently:
<pre>
rank-profile dot_product {
first-phase {
expression {
reduce(
join(
query(tensor),
attribute(tensor_attribute),
f(x,y)(x * y)
),
sum
)
}
}
}
</pre>
...and represents the general dot product for tensors of any order.
Details about tensor ranking functions including lambda expression and available aggregators
can be found in the <a href="reference/tensor.html">tensor reference documentation</a>.
More examples of tensor expression can be found in the <a href="tensor-intro.html">tensor introduction</a>.
</p>
<h2 id="constant-tensors">Constant tensors</h2>
<p>
In addition to document tensors and query tensors,
<a href="reference/search-definitions-reference.html#constant">constant tensors</a>
can be put in the <a href="reference/application-packages-reference.html">application package</a>.
This is useful when constant tensors are used in ranking expressions,
for instance machine learned models. Example from the
<a href="https://github.com/vespa-engine/sample-apps/blob/master/basic-search-tensor/src/main/application/searchdefinitions/music.sd#L43-L46">sample app</a>:
<pre>
constant tensor_constant {
file: constants/constant_tensor_file.json
type: tensor(x{})
}
</pre>
This defines a new tensor rank feature with the type as defined and the
contents distributed with the application package in the file
<em>constants/constant_tensor_file.json</em>. The format of this file is the
<a href="reference/document-json-put-format.html#tensor">tensor JSON format</a>, it can be compressed,
see the <a href="reference/search-definitions-reference.html#constant">reference</a> for examples.
</p><p>
To use this tensor in a rank expression, encapsulate the constant name with <code>constant(...)</code>:
<pre>
rank-profile use_constant_tensor {
first-phase {
expression: sum(query(tensor) * attribute(tensor_attribute) * constant(tensor_constant))
}
}
</pre>
The above expression combines three tensors: the query tensor, the document tensor and a constant tensor.
</p>
<!-- Consider having a section on the Java API here - links to javadoc in heading now
<h2 id="tensor-java-api">Tensor Java API</h2> -->
<h2 id="use-cases">Use cases</h2>
<p>
In the following section, find common use cases that can be solved using tensor operations.
</p>
<h3 id="dot-product-between-query-and-document-vectors">Dot Product between query and document vectors</h3>
<p>
Assume we have a set of documents where each document contains a vector of size 4.
We want to calculate the dot product between the document vectors and a vector passed down with the query
and rank the results according to the dot product score.
</p><p>
The following sd-file defines an attribute tensor field
with a tensor type that has one indexed dimension <code>x</code> of size 4.
In addition we define a rank profile that calculates the dot product.
<pre>
search example {
document example {
field document_vector type tensor(x[4]) {
indexing: attribute | summary
attribute: tensor(x[4])
}
}
rank-profile dot_product {
first-phase {
expression: sum(query(query_vector)*attribute(document_vector))
}
}
}
</pre>
The tensor to pass down with query is defined in a query profile type with the same tensor type as the field in the document:
<pre>
&lt;query-profile-type id="myProfileType"&gt;
&lt;field name="ranking.features.query(query_vector)" type="tensor(x[4])" /&gt;
&lt;/query-profile-type&gt;
</pre>
Example document with the vector [1.0, 2.0, 3.0, 5.0]:
<pre>
[
{ "put": "id:example:example::0", "fields": {
"document_vector" : {
"cells": [
{ "address" : { "x" : "0" }, "value": 1.0 },
{ "address" : { "x" : "1" }, "value": 2.0 },
{ "address" : { "x" : "2" }, "value": 3.0 },
{ "address" : { "x" : "3" }, "value": 5.0 }
]
}
}
}
]
</pre>
Example query set in a searcher with the vector [1.0, 2.0, 3.0, 5.0]:
<pre>
public Result search(Query query, Execution execution) {
query.getRanking().getFeatures().put("query(query_vector)",
Tensor.Builder.of(TensorType.fromSpec("tensor(x[4])")).
cell().label("x", 0).value(1.0).
cell().label("x", 1).value(2.0).
cell().label("x", 2).value(3.0).
cell().label("x", 3).value(5.0).build());
return execution.search(query);
}
</pre>
</p>
<h3 id="matrix-product-between-1d-vector-and-2d-matrix">Matrix Product between 1d vector and 2d matrix</h3>
<p>
Assume we have a 3x2 matrix represented in an attribute tensor field <code>document_matrix</code>
with a tensor type <code>tensor(x[3],y[2])</code> with content:
<pre>
{ {x:0,y:0}:1.0, {x:1,y:0}:3.0, {x:2,y:0}:5.0, {x:0,y:1}:7.0, {x:1,y:1}:11.0, {x:2,y:1}:13.0 }
</pre>
Also assume we have 1x3 vector passed down with the query as a tensor
with type <code>tensor(x[3])</code> with content:
<pre>
{ {x:0}:1.0, {x:1}:3.0, {x:2}:5.0 }
</pre>
that is set as <code>query(query_vector)</code> in a searcher
as specified in <a href="reference/tensor.html#query-feature">query feature</a>.
</p><p>
To calculate the matrix product between the 1x3 vector and 3x2 matrix (to get a 1x2 vector)
use the following ranking expression:
<pre>
sum(query(query_vector) * attribute(document_matrix),x)
</pre>
This is a sparse tensor product over the shared dimension <code>x</code>,
followed by a sum over the same dimension.
</p>