Vespa is an engine for low-latency computation over large data sets.
Switch branches/tags
7-java11 7 andreer/allocation-improvements aressem/update-vespa-version-2 arnej/sort-missing-values-take-2 balder/add-chunked-file-rpc-skeleton balder/add-gc-details balder/add-uri-support-to-tensor-constants balder/allocate-new-fieldvalue-no-more-reuse-rebased-1 balder/apply-termwise-filters-on-match-phase-2 balder/apply-termwise-filters-on-match-phase balder/balder-arnej-default-down balder/bytebuffer-2-nbostream balder/cancel-unhealty-correlation-matchphase balder/diverisity-in-first-phase balder/enable-assert-once-per-release-for-entryref balder/enable-glibc-lock-elision balder/ensure-exceptions-are-caught-and-errors-propagated balder/ensure-status-is-known-at-construction-time balder/faster-computation-of-residue balder/faster-simple-struct balder/flip-bits-in-hash-method balder/less-validation-for-struct-fields balder/reply-in-own-thread balder/ressurect-docstore-validation-rebased-1 balder/run-verify-ranksetup-as-jni balder/simpler-and-safer-hash balder/use-xxhash32 bjorncs/tls bratseth/json-rendering-of-dynamic-summary-fields bratseth/send-result-on-exception freva/extend-node-agent-context freva/modify-callback geirst/fix-bug-in-datastore-memory-allocation geirst/7-remove-enable-multiple-bucket-spaces-flag gjoranv/allow-adding-config-structs gjoranv/attach-javadocs gjoranv/java9-prep-internal gjoranv/java9-prep-internal3 gjoranv/jersey-2.26 gjoranv/jersey-2.27 gjoranv/jersey-2.27_2 gjoranv/new-metrics-handler gv7/Java11_new gv7/component-cleanup gv7/qr-templates-cleanup gv/7 gv/7_05dec gv/7_10oct gv/7_21nov hakonhall/feature-flags hakonhall/swap-version-and-release hmusum/add-zone hmusum/avoid-index-out-of-bounds hmusum/remove-cache-in-NodeIdentifier jvenstad/allow-emergency-deployments-through-controller jvenstad/autolocking-proposition jvenstad/fix-node-repo-cache jvenstad/freeze-to-block-window jvenstad/increase-feed-client-close-timeout jvenstad/utils master ollivir/round-minimum-coverage-up revert-7200-havardpe/discard-transport-events-more-safely revert-7487-revert-7482-balder/obey-timeout-too revert-7798-balder/swap-buffer-and-offset-bits revert-7876-toregge/process-positon-summary-entries rpmbuild smorgrav/systemname smorgrav/tenantpiperestapi toregge/compact-lid-space-on-source-selector toregge/detect-nul-byte-in-word toregge/fake-old-bucketspaces-config vekterli/make-test-script-python3-compatible vespa-java
Nothing to show
Clone or download
Latest commit 1cb3a4e Dec 14, 2018
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
abi-check-plugin Use mockito-core instead of mockito-all. Dec 3, 2018
annotations Replace 'tonytv' with full name in author tags Jul 5, 2018
application-deploy-plugin Remove explicit versions - inherit from parent. Jun 29, 2018
application-model Revert "Revert "Add infrastructure applications to DuperModel"" Dec 3, 2018
application-preprocessor Use maven-shade-plugin to assemble fat jar Oct 19, 2018
application Add Exceptions.findCause() Dec 7, 2018
athenz-identity-provider-service Cleanup Nov 20, 2018
bundle-plugin-test Enable and fix test to verify bundle version. Sep 13, 2018
bundle-plugin Ignore test to verify no stackoverflow error is thrown. Nov 2, 2018
chain Java 9: Class.newInstance() is deprecated. May 3, 2018
clustercontroller-apps Remove experimental enable-multiple-bucket-spaces flag. Nov 23, 2018
clustercontroller-apputil Remove explicit maven-compiler-plugin config. Inherit from parent. Apr 25, 2018
clustercontroller-core Remove experimental enable-multiple-bucket-spaces flag. Nov 23, 2018
clustercontroller-standalone Use maven-shade-plugin to assemble fat jar Oct 19, 2018
clustercontroller-utils Revert "Revert "Revert "Revert "Enforce CC timeouts in Orchestrator 4… Nov 1, 2018
component Deprecate com.yahoo.config.provision.Version Dec 1, 2018
config-application-package Deprecate com.yahoo.config.provision.Version Dec 1, 2018
config-bundle Remove Scala remnants from Maven build files Jun 25, 2018
config-class-plugin Replace 'tonytv' with full name in author tags Jul 5, 2018
config-lib Use full name Oct 3, 2018
config-model-api Remove unused constructor Dec 7, 2018
config-model-fat Import ai.vespa.rankingexpression.importer.configmodelview Nov 27, 2018
config-model Support for configuring and feeding float16 and bool field types Dec 14, 2018
config-provisioning Deprecate com.yahoo.config.provision.Version Dec 1, 2018
config-proxy Use shade plugin instead of assembly. Sep 12, 2018
config Include stack trace when constructed in log warning Dec 4, 2018
config_test Add copyright header Oct 27, 2017
configd - NULL -> nullptr Sep 27, 2018
configdefinitions Support for configuring and feeding float16 and bool field types Dec 14, 2018
configgen Add build() method to config builders Nov 30, 2018
configserver Inject FlagSource through component registry Dec 13, 2018
configutil use portal server for unit test Dec 13, 2018
container-accesslogging Minor fixes Oct 14, 2018
container-core Merge pull request #7869 from vespa-engine/bratseth/less-use-of-conta… Dec 5, 2018
container-dependencies-enforcer Update jackson to 2.8.11 and jackson-databind to 2.8.11.2 Sep 19, 2018
container-dependency-versions Upgrade Jetty to 9.4.14 Nov 19, 2018
container-dev Separate model integration into a separate module Nov 21, 2018
container-di Revert "Merge pull request #7837 from vespa-engine/revert-7836-bratse… Dec 2, 2018
container-disc Less use of Container Dec 4, 2018
container-integration-test Add missing newline. Jul 12, 2018
container-jersey2 Replace 'tonytv' with full name in author tags Jul 5, 2018
container-messagebus GC unused imports. Oct 25, 2018
container-search-and-docproc Split parent + container-dependency-versions from root pom. Dec 1, 2017
container-search-gui Add search api reference temporarily to avoid download during build Oct 23, 2018
container-search Support for configuring and feeding float16 and bool field types Dec 14, 2018
container-test-jars Remove Scala remnants from Maven build files Jun 25, 2018
container-test Jdisc provides bouncycastle now, so don't hide it. Mar 7, 2018
container Separate model integration into a separate module Nov 21, 2018
controller-api cost resources by property (#7789) Dec 10, 2018
controller-server Change response from application api as well — it's the go-forward Dec 14, 2018
defaults reorder to get all defaults in the same place Jul 19, 2018
dist Use llvm 7 on Fedora 29 and Fedora 30. Aug 29, 2018
docker-api REMOVING is also a valid state Dec 7, 2018
docker Fixed typo in environment variable assignment Mar 26, 2018
docproc Do not create static stuff you do not use. Dec 4, 2018
docprocs Use hashmap for faster access. Oct 25, 2018
document Support for configuring and feeding float16 and bool field types Dec 14, 2018
documentapi Nonfunctional changes only Dec 13, 2018
documentgen-test Revert "Revert "Always output tensor type in toString"" Nov 29, 2018
eval Merge pull request #6518 from vespa-engine/havardpe/avoid-old-style-v… Aug 8, 2018
fastlib add and use HDR_ABORT instead Jun 20, 2018
fastos Clean up unused code Nov 14, 2018
fat-model-dependencies Include model evaluation Aug 31, 2018
fbench use crypto engine and sync crypto socket in http client Oct 12, 2018
fileacquirer Use fileAcquirer Sep 10, 2018
filedistribution Use maven-shade-plugin to assemble fat jar Oct 19, 2018
flags Revert "Revert "Add infrastructure applications to DuperModel"" Dec 3, 2018
fnet use generic latch Nov 30, 2018
fsa Minor fixes Oct 14, 2018
indexinglanguage Update to latest ph-javacc-maven-plugin. Dec 6, 2018
jaxrs_client_utils Remove unused import Nov 2, 2018
jaxrs_utils Split parent + container-dependency-versions from root pom. Dec 1, 2017
jdisc-security-filters Allow okta headers Oct 22, 2018
jdisc_core Less use of Container Dec 4, 2018
jdisc_core_test Add copyright header Oct 1, 2018
jdisc_http_service Revert "Add support for connection throttling in JDisc" Dec 12, 2018
jdisc_jetty Remove aries.spifly and asm-debug-all. Jun 6, 2018
jdisc_messagebus_service GC unused imports. Oct 25, 2018
jrt Nonfunctional changes only Dec 13, 2018
jrt_test Make jrt_test see transitive dependencies of jrt Oct 19, 2018
juniper Use template args for the class, not on the find method. Nov 1, 2018
linguistics Deprecated methods and add OptimaizeDetector Nov 1, 2018
logd Add missing reference & Oct 9, 2018
logforwarder forgot terminating nullptr Jul 27, 2018
logserver Minor fixes Oct 14, 2018
lowercasing_test Update copyright headers Jun 14, 2017
maven-plugins Fix whitespace. Nov 30, 2018
messagebus-disc fixup! Use maven-shade-plugin to assemble fat jar Oct 19, 2018
messagebus Nonfunctional changes only Dec 13, 2018
messagebus_test Remove the costly and unnecessary setSlobrokId interface. Oct 4, 2018
metrics Use template args for the class, not on the find method. Nov 1, 2018
model-evaluation Move all importing to model-integration Nov 22, 2018
model-integration Upgrade protoc jar maven plugin version Dec 4, 2018
node-admin Remove outdated comment Dec 7, 2018
node-maintainer Remove java dep for maintainer RPM Nov 19, 2018
node-repository Simplify infrastructure applications Dec 7, 2018
orchestrator-restapi Avoid stacktrace in log on timeouts Nov 9, 2018
orchestrator Avoid stacktrace in log on timeouts Nov 9, 2018
parent Update to latest ph-javacc-maven-plugin. Dec 6, 2018
persistence Support test-and-set for auto-create document updates Nov 26, 2018
persistencetypes clean up remnants of .mak files Jun 11, 2018
predicate-search-core Non-functional changes only Oct 2, 2018
predicate-search Non-functional changes only Oct 2, 2018
processing Revert "Merge pull request #7094 from vespa-engine/revert-7070-bratse… Sep 25, 2018
protocols Remove never used per storage node ops latencies in host info. Mar 13, 2018
provided-dependencies Jdisc provides bouncycastle now, so don't hide it. Mar 7, 2018
searchcommon Use explicit EntryRef constructor to avoid unintended conversion Nov 28, 2018
searchcore Fix typos and remove a superfluous line Dec 14, 2018
searchcorespi - Decouple index config from ProtonConfig. Dec 3, 2018
searchlib Merge pull request #7896 from vespa-engine/geirst/fix-bug-in-datastor… Dec 6, 2018
searchsummary Handle imported attributes in AttributeCombinerDFW. Nov 26, 2018
security-utils Change LOG_ONLY config value from 'log-only' to 'log_only' Dec 10, 2018
service-monitor Monitor health of host infra applications Dec 8, 2018
serviceview Add GET suspended status to application/v2 Oct 22, 2018
simplemetrics Move all importing to model-integration Nov 22, 2018
slobrok Expose exception information. If totally unknown, generate core. Oct 9, 2018
socket_test Fix author tag for Simon Jul 5, 2018
staging_vespalib Use a directory that the vespa user has control over. Dec 7, 2018
standalone-container Revert "Merge pull request #7837 from vespa-engine/revert-7836-bratse… Dec 2, 2018
statistics Minor fixes Oct 14, 2018
storage Support test-and-set for auto-create document updates Nov 26, 2018
storageapi Remove experimental enable-multiple-bucket-spaces flag. Nov 23, 2018
storageframework remove legacy tags, always pass vector Oct 8, 2018
storageserver Remove the costly and unnecessary setSlobrokId interface. Oct 4, 2018
streamingvisitors Pass stringref by value Aug 10, 2018
tenant-base Separate jar with tests to scan to avoid Surefire breakdown Nov 2, 2018
tenant-cd Add copyright header Oct 1, 2018
testutil Support file templates Jan 27, 2018
travis Update to reflect changes in docker-image-dev. Nov 13, 2018
vagrant Merge pull request #6983 from vespa-engine/bjorncs/vagrant Sep 17, 2018
vbench clean up remnants of .mak files Jun 11, 2018
vdslib getContentType -> contentStruct and java doc. Nov 1, 2018
vdstestlib clean up remnants of .mak files Jun 11, 2018
vespa-application-maven-plugin Replace plugin used for a single mkdir with mkdirs in dependent Sep 20, 2018
vespa-athenz Get access token from request attribute Nov 20, 2018
vespa-documentgen-plugin Deprecating a whole lot of body/header related methods that should ha… Oct 31, 2018
vespa-hadoop Do NOT specify custom version of maven plugins in child modules. May 3, 2018
vespa-http-client Specify type Dec 6, 2018
vespa_feed_perf Use maven-shade-plugin to assemble fat jar Oct 19, 2018
vespa_fsa Publish Jun 15, 2016
vespa_jersey2 Split parent + container-dependency-versions from root pom. Dec 1, 2017
vespa_qrs Publish Jun 15, 2016
vespabase Honor JAVA_HOME even if java is in PATH Nov 14, 2018
vespaclient-container-plugin Add and use cross-cluster bucket space config Dec 7, 2018
vespaclient-core Nonfunctional changes only Dec 13, 2018
vespaclient-java Adjust command line parsing for vespa-get to handle priority names. Nov 5, 2018
vespaclient Remove the costly and unnecessary setSlobrokId interface. Oct 4, 2018
vespajlib Support for configuring and feeding float16 and bool field types Dec 14, 2018
vespalib Merge pull request #7919 from vespa-engine/vekterli/support-certifica… Dec 11, 2018
vespalog go back to old way of computing filename Nov 5, 2018
vespamalloc If there are intermediate errors that has no effect on the end result… Sep 7, 2018
vsm Support for configuring and feeding float16 and bool field types Dec 14, 2018
yolean Add Exceptions.findCause() Dec 7, 2018
zkfacade Revert "Revert "No need for restricting access to zookeeper in hosted… Oct 24, 2018
.gitignore Ignore clion files. Oct 6, 2017
.travis.yml Don't delete com.yahoo.athenz artifacts Nov 7, 2017
CMakeLists.txt Add flags to top-level CMakeLists.txt Dec 3, 2018
CONTRIBUTING.md Add copyright header Oct 27, 2017
Code-of-Conduct.md Add code of conduct Dec 6, 2018
LICENSE Publish Jun 15, 2016
OWNERS Publish Jun 15, 2016
README-cmake.md Add copyright header Oct 27, 2017
README.md Add license header section Dec 7, 2018
TODO.md Add copyright header Oct 27, 2017
VERSION Enable new storage api protocol serialization. May 4, 2018
bootstrap-cmake.sh Reduce duplication in cmake command args. Sep 17, 2018
bootstrap-cpp.sh Add option to setup unprivileged compilation of Vespa. Sep 14, 2018
bootstrap.sh Use multiple threads for build with multiple modules Dec 11, 2018
build_settings.cmake Default now 6.0 unless set explicit. Aug 12, 2018
cppunit-parallelize.py Update copyright headers Jun 14, 2017
dist.sh Set version number when building Vespa RPM. Bump minor number in VERS… Aug 28, 2018
empty.cpp Publish Jun 15, 2016
functions.cmake Add make function to install absolute target symlink. Oct 8, 2018
pom.xml Merge pull request #7760 from vespa-engine/hakonhall/new-flags-module Nov 26, 2018
valgrind-suppressions.txt Suppress memory leak in setenv. Jun 7, 2018
vtag.cmake Update copyright headers Jun 14, 2017

README.md

#Vespa

The big data serving engine - Store, search, rank and organize big data at user serving time. Vespa is an engine for low-latency computation over large data sets. It stores and indexes your data and executes distributed queries including evaluation of machine-learned models over many data points in real time.

Travis-CI build status: Build Status

Table of contents

Background

Use cases such as search, recommendation and personalization need to select a subset of data in a large corpus, evaluate machine-learned models over the selected data, organize and aggregate it and return it, typically in less than 100 milliseconds, all while the data corpus is continuously changing.

This is hard to do, especially with large corpuses that needs to be distributed over multiple nodes and evaluated in parallel. Vespa is a platform which performs these operations for you. It has been in development for many years and is used on a number of large internet services and apps which serve hundreds of thousands of queries from Vespa per second.

Install

To get started using Vespa pick one of the quick start documents:

Usage

Full documentation is available on https://docs.vespa.ai.

Contribute

We welcome contributions! See CONTRIBUTING.md to learn how to contribute.

If you want to contribute to the documentation, see https://github.com/vespa-engine/documentation

Building

You do not need to build Vespa to use it, but if you want to contribute you need to be able to build the code. This section explains how to build and test Vespa.

Set up the build environment

C++ and Java building is supported on CentOS 7. The Java source can also be built on any platform having Java 8 and Maven installed. We recommend using the following environment: Create C++ / Java dev environment on CentOS using VirtualBox and Vagrant. You can also setup CentOS 7 natively and install the following build dependencies:

sudo yum-config-manager --add-repo https://copr.fedorainfracloud.org/coprs/g/vespa/vespa/repo/epel-7/group_vespa-vespa-epel-7.repo
sudo yum -y install epel-release centos-release-scl yum-utils
sudo yum -y install ccache \
    rpm-build
yum-builddep -y <vespa-source>/dist/vespa.spec

Build Java modules

export MAVEN_OPTS="-Xms128m -Xmx1024m"
source /opt/rh/rh-maven35/enable
bash bootstrap.sh java
mvn -T <num-threads> install

Build C++ modules

Replace <build-dir> with the name of the directory in which you'd like to build Vespa. Replace <source-dir> with the directory in which you've cloned/unpacked the source tree.

bash bootstrap-cpp.sh <source-dir> <build-dir>
cd <build-dir>
make -j <num-threads>
ctest3 -j <num-threads>

Create RPM packages

sh dist.sh VERSION && rpmbuild -ba ~/rpmbuild/SPECS/vespa-VERSION.spec

License

Code licensed under the Apache 2.0 license. See LICENSE for terms.