Repository navigation
4 Robot Registration
Once a robot has been implemented, it must be deployed so that it can communicate with the DESTINY repository and receive enhancement batches.
Deploying a robot consists of three steps:
- Registering the robot with the DESTINY repository.
-
Configuring the robot's environment by setting the required values in its
.envfile. - Assigning work to the robot. This can either be done by registering enhancement requests for specific references or by configuring an automated query that continuously creates enhancement batches for the robot.
The following sections describe these steps in more detail.
Before a robot can communicate with the DESTINY repository, it must be registered with a robot name, robot owner and a description. A robot name must be unique at the DESTINY repository. So a robot name like query-robotmost likely will not work.
To register a robot, run the following command from the destiny-repository folder:
uv run python -m cli.register_robot \
--name "<ROBOT NAME>" \
--owner "<ROBOT OWNER>" \
--description "<ROBOT DESCRIPTION>" \
--env stagingThe command opens a browser window and prompts you to authenticate with your DESTINY credentials. Your user account must have the robot.writer role to register a robot. If you do not have this role, please contact the DESTINY repository team and ask them to grant you the required permission.
After successful registration, the repository returns a unique robot ID and client secret. These credentials are displayed only once. Make sure to store them securely immediately after registration.
The robot ID and client secret identify the robot and are required when the robot communicates with the DESTINY repository, for example when polling for enhancement batches or uploading results. They should therefore be stored in the robot's .env file.
The robot's .env file contains the configuration and credentials required to communicate with the DESTINY repository.
The relevant entries are:
DESTINY_REPOSITORY_URL=http://127.0.0.1:8000
ROBOT_ID=<your robot id>
ROBOT_SECRET=<your robot secret>
ENV=local < development / staging / production >The ROBOT_ID and ROBOT_SECRET are obtained when the robot is registered.
The ENV variable determines which DESTINY repository environment the robot communicates with. When ENV is set to local, DESTINY_REPOSITORY_URL specifies the repository instance to connect to. For the other environments, the robot connects to the corresponding environment provided by the DESTINY repository team, and DESTINY_REPOSITORY_URL is ignored.
This does not work with base_robot, so you need to explicitly set the correct DESTINY_REPOSITORY_URL in the .env file.
| Development | https://api.dev.evidence-repository.org |
|---|---|
| Staging | https://api.staging.evidence-repository.org |
| Production | https://api.evidence-repository.org |
The DESTINY repository team currently recommends using the staging environment.
A registered robot will only receive work if enhancement batches are created for it. The DESTINY repository supports several mechanisms for assigning work.
For the robot-facing endpoints used during normal operation, authentication can generally be performed using HMAC authentication. The robot authenticates using its robot ID and robot secret.
There are currently three relevant ways of creating enhancement requests.
Note: The whole thing is currently under development. Please watch out for the following issues: 884
The first approach is to register an automation query or percolator query for a robot using the POST /v1/enhancement-requests/automations/ endpoint.
Whenever new references matching the query become available, the repository automatically creates enhancement batches for the associated robot. This mechanism is particularly useful for continuously running robots that should process all newly arriving references matching specific criteria.
For example, a relevance classifier can be configured to automatically process all references that have previously received the annotation scheme="query:inclusion", label="cdr", and value=true. The following percolator query registers such an automation:
{
"bool": {
"must": [
{
"nested": {
"path": "changeset.enhancements.content.annotations",
"query": {
"bool": {
"must": [
{
"term": {
"changeset.enhancements.content.annotations.scheme": "query:inclusion"
}
},
{
"term": {
"changeset.enhancements.content.annotations.label": "cdr"
}
},
{
"term": {
"changeset.enhancements.content.annotations.value": true
}
}
]
}
}
}
}
]
}
}Other examples can be found in the DESTINY documentation.
At present, automated queries do not support free-text searches. As a consequence, query robots that identify references using text-based search expressions cannot currently be automated through this mechanism. Support for text-based automated queries is planned and is being tracked in Issue #884.
Registering Enhancement Search Requests via /v1/enhancement-requests/search/
A second approach is to search for references via POST /v1/enhancement-requests/search/and register an enhancement request for a specific robot.
The endpoint accepts a Lucene search query together with optional annotation filters and is therefor suitable for query robots. The annotation filters allow the search to be restricted to references with particular annotations. For example, an annotation can be specified by its scheme, label, and optionally a minimum score:
<scheme>[/<label>][@<score>]For example, to find references containing the term pyro in the title or abstract and having an annotation from the cdr-technology scheme with the label biochar and a score of at least 0.9, the request would look like:
POST /v1/references/search/?q=pyro&annotation=cdr-technology/biochar@0.9The q parameter contains the Lucene search query, while the annotation parameter applies an additional annotation filter. Since the score specifies a minimum, @0.9 matches annotations with a score of 0.9 or higher.
This endpoint also accepts negated annotations, which can be used to search for references that do not contain a specific annotation. For example:
POST /v1/references/search/?q=pyro%20AND%20NOT%20annotations:%22query-inclusion/cdr%22The third approach is to explicitly request enhancements for a set of references using the POST /v1/enhancement-requests/ endpoint and searching the complete DESTINY repository.
In this case, the request contains the reference IDs that should be processed together with the target robot. The repository creates the corresponding enhancement batches, which can then be retrieved by the robot during polling.
Reference IDs can be obtained using the GET /v1/references/search/ endpoint. Unlike the automation mechanism, this endpoint also supports free-text search. Note: This endpoint uses OAuth identification.
Explicit enhancement requests are also the only mechanism for processing references that already exist in the repository. Automation queries are evaluated only on newly created or modified changesets and therefore do not trigger for existing references that have not changed.
A typical workflow is to search for references matching a free-text query while excluding those that have already received the desired enhancement. The resulting reference IDs are then submitted as an enhancement request. For large result sets, the search must be performed in multiple requests using pagination, as Elasticsearch limits search results to a maximum of 10,000 documents per query.
A script which handles this workflow can be found here .
The DESTINY-repository supports two authentication methods: HMAC authentication, using a robot ID and robot secret, and OAuth authentication, which requires you to enter your user credentials in a browser window once per open terminal every 12 hours.
Many API endpoints support both authentication methods. Robots should use HMAC authentication, as it is intended for machine-to-machine access.
The destiny-sdk provides convenient clients and authentication handlers for both methods. See the SDK documentation for details on OAuth and HMAC authentication.
from destiny_sdk.client import OAuthClient
# Either use env for defaults or provide explicit auth and base_url
client = OAuthClient(env="production")
# OR
client = OAuthClient(
base_url="https://destiny-repository.example.com",
auth=OAuthMiddleware(...),
)from uuid import UUID
import httpx
from destiny_sdk.client import HMACSigningAuth
hmac_auth = HMACSigningAuth(
secret_key=robot_secret,
client_id=UUID(robot_id),
)
client = httpx.Client(
base_url=repository_url,
auth=hmac_auth,
timeout=30.0,
)This script creates enhancement requests creates enhancement requests for an existing robot. It performs a search for matching references and submits the resulting reference IDs to the DESTINY repository as enhancement requests. Run it with
uv run python create_enhancement_requests.py config.jsonwhere config.json defines the behavior of the script, see an example here.