Extends the album-recommendations sample application with a Searcher component in Java which does query and result processing. Refer to developing searchers for more information.
This sample app introduces how to write system tests and how to integrate with a CI/CD pipeline.
See getting started for troubleshooting.
Security notes:
- To deploy to your instance, a personal deploy key is required. See step 4 below.
- To read and write to the instance's endpoint, a certificate and a private key is required. See step 2 below. Find more details in Data Plane, see Client certificate.
- Instead of using a personal deploy key, one can also deploy using the console, see step 5 in ../album-recommendation/README.md.
Prerequisites: git, Java 11, mvn 3.6.1 and openssl.
-
Download sample apps:
$ git clone https://github.com/vespa-engine/sample-apps.git && cd sample-apps/album-recommendation-java
-
Get a X.509 certificate and private key. Create a self-signed certificate / private key:
$ openssl req -x509 -nodes -days 14 -newkey rsa:4096 \ -subj "/C=NO/ST=Trondheim/L=Trondheim/O=My Company/OU=My Department/CN=example.com" \ -keyout data-plane-private-key.pem -out data-plane-public-cert.pem -
Add certificate to application package (it must be copied as clients.pem):
$ mkdir -p src/main/application/security && cp data-plane-public-cert.pem src/main/application/security/clients.pem -
Go to http://console.vespa.ai/, choose tenant and click Keys to generate and save the personal API key. The key is saved to
$HOME/Downloads/TENANTNAME.pem. Then click "Create application" -
Edit the properties
tenantandapplicationinpom.xml— use the values entered in the console in 4. -
Build the app:
$ mvn clean package
-
Deploy the application to the
devenvironment and wait for it to start - update theapiKeyFileto be the file you downloaded above:$ mvn vespa:deploy -DapiKeyFile=$HOME/Downloads/TENANTNAME.pem -
Now is a good time to read http://cloud.vespa.ai/automated-deployments, as first time deployments takes a few minutes. Seeing CERTIFICATE_NOT_READY / PARENT_HOST_NOT_READY / LOAD_BALANCER_NOT_READY is normal. The endpoint URL is printed in the Install application section when the deployment is successful - copy this for the next step.
-
Store the endpoint of the application:
$ ENDPOINT=https://end.point.name
Try the endpoint:
$ curl --cert data-plane-public-cert.pem --key data-plane-private-key.pem $ENDPOINT -
Feed documents:
$ curl --cert data-plane-public-cert.pem --key data-plane-private-key.pem \ -H "Content-Type:application/json" --data-binary @src/test/resources/A-Head-Full-of-Dreams.json \ $ENDPOINT/document/v1/mynamespace/music/docid/1 $ curl --cert data-plane-public-cert.pem --key data-plane-private-key.pem \ -H "Content-Type:application/json" --data-binary @src/test/resources/Love-Is-Here-To-Stay.json \ $ENDPOINT/document/v1/mynamespace/music/docid/2 $ curl --cert data-plane-public-cert.pem --key data-plane-private-key.pem \ -H "Content-Type:application/json" --data-binary @src/test/resources/Hardwired...To-Self-Destruct.json \ $ENDPOINT/document/v1/mynamespace/music/docid/3
-
Visit documents:
$ curl --cert data-plane-public-cert.pem --key data-plane-private-key.pem \ "$ENDPOINT/document/v1/mynamespace/music/docid?wantedDocumentCount=100" -
Recommend albums, send user profile in query:
$ curl --cert data-plane-public-cert.pem --key data-plane-private-key.pem \ "$ENDPOINT/search/?ranking=rank_albums&yql=select%20%2A%20from%20sources%20%2A%20where%20sddocname%20contains%20%22music%22%3B&ranking.features.query(user_profile)=%7B%7Bcat%3Apop%7D%3A0.8%2C%7Bcat%3Arock%7D%3A0.2%2C%7Bcat%3Ajazz%7D%3A0.1%7D"Limit to albums with the term "to" in title:
$ curl --cert data-plane-public-cert.pem --key data-plane-private-key.pem \ "$ENDPOINT/search/?ranking=rank_albums&yql=select%20%2A%20from%20sources%20%2A%20where%20album%20contains%20%22to%22%3B&ranking.features.query(user_profile)=%7B%7Bcat%3Apop%7D%3A0.8%2C%7Bcat%3Arock%7D%3A0.2%2C%7Bcat%3Ajazz%7D%3A0.1%7D" -
At this point, the application is built, unit tested, deployed to a dev instance, fed to and a few test queries have been run. Safe deployments depends on automated testing. Vespa Cloud has support for running system and staging tests for every change to an application. These tests are run as JUnit tests, but use the endpoints of a real deployment of the application. When submitting an application to Vespa Cloud, a test instance is set up and tests automatically run using its endpoints. To develop system and staging tests, deploy the application to dev (like above) and run tests like ExampleSystemTest:
$ mvn test -Dtest.categories=system \ -DdataPlaneKeyFile=data-plane-private-key.pem \ -DdataPlaneCertificateFile=data-plane-public-cert.pem \ -DapiKeyFile=$HOME/Downloads/TENANTNAME.pem
or run it directly from an IDE. ai.vespa.hosted.cd.Endpoint must have access to the data plane key and certificate pair, to talk to the application endpoint.
Find more details in the Vespa Cloud API and automated-deployments.
-
To run tests against a deployment running in Docker on localhost (instead of using dev), configure endpoint location:
{ "localEndpoints": { "container": "http://localhost:8080/" } }in some file
/path/to/test/config, and run JUnit tests with-Dvespa.test.config=/path/to/test/config -Dtest.categories=system. Refer to album-recommendation-selfhosted for how to create the application package. -
When System and Staging tests are ready, deploy to production. Command to build and submit application to the hosted Vespa API is
mvn clean vespa:compileVersion mvn -P fat-test-application \ -Dvespaversion="$(cat target/vespa.compile.version)" \ -DauthorEmail=<span style="{background-color: yellow;}">user@domain</span> \ package vespa:submitTo track versions through the pipeline, assuming you're using
gitfor version control, you can instead specify-Drepository=$(git config --get remote.origin.url) -Dbranch=$(git rev-parse --abbrev-ref HEAD) -Dcommit=$(git rev-parse HEAD) -DauthorEmail=$(git log -1 --format=%aE)