Writing Tests for an Authority
The purpose of testing is to determine the health of the system and its ability to return accurate results within a reasonable amount of time.
Feel free to just follow the pattern of writing a test as described in this file. Note that indentation matters, so be sure your tests indent the same amount as the tests already in the file. If you want to dig in and understand YAML, the following resources may be helpful (listed from most likely to be helpful to more complex).
There are two services provided by QaServer.
-
- A query string is passed to the search facility of the authority and a set of potential matching terms are returned. -
- The URI or ID of a specific term is passed to the authority and all available data for the term is returned.
- Connection Test - Can I get data from the system now?
- Accuracy Test - Do I get the expected data?
- Performance Test - How long does it take to get data?
Location of File: lib/generators/qa_server/templates/config/authorities/linked_data/scenarios
Filename: authority + service + 'validation.yml' (e.g. geonames_ld4l_cache_validation.yml)
Deciding the service value to use...
- ld4l_cache - For most authorities, the data will be cached in services.ld4l.org system. Use this as the service to identify that the data is coming from services.ld4l.org cache.
- direct - If data will be accessed directly from the authority using the authority provided API, then use this as the service to identify that the data is coming directly from the external authority.
* Required
| Field | Value Type | Example Value | Comments | |
|---|---|---|---|---|
| * | authority | no value | The following will define characteristics that apply to all scenarios. | |
| * | service | string | ld4l_cache | Which service is providing the data. |
NOTE: service: must be indented 2 spaces on the line after authority: which has no indentation.
Example:
All files should initially be created with the following as the starting point for the file. The only thing that might change is the name of the service. Typically, this will be ld4l_cache or direct. And most commonly, it will be ld4l_cache.
---
authority:
service: ld4l_cache
To see the authority: section in context, explore any of the validation files in the scenarios directory.
Purpose: Determine if QA can retrieve data at this moment in time from an authority.
Used to Test Services:
* Required
| Field | Value Type | Example Value | Comments | |
|---|---|---|---|---|
| * | query | string | animation | |
| subauth | string | organization | The supported subauths are defined in the configuration file for the authority. |
Example Tests:
Run a query for 'animation'. If sub-authorities are supported by the authority, it will search all sub-authorities.
-
query: animation
Run a query for 'animation' searching only the 'intangible' sub-authority.
-
query: animation
subauth: intangible
* Required
| Field | Value Type | Example Value | Comments | |
|---|---|---|---|---|
| * | identifier | string | "http://id.loc.gov/authorities/genreForms/gf2011026141" | URI or ID |
Example Tests:
Fetch the term with specified URI
-
identifier: "http://id.loc.gov/authorities/genreForms/gf2011026141"
Fetch the term with specified ID
-
identifier: '1914919'
Purpose: Determine if a search for a specific term has the desired result within the first few search results.
Used to Test Services:
* Required
| Field | Value Type | Example Value | Comments | |
|---|---|---|---|---|
| * | query | string | animation | |
| subauth | string | organization | The supported subauths are defined in the configuration file for the authority. | |
| * | position | integer | 10 | The subject_uri must be in the results by position for this test to pass. |
| * | subject_uri | string | "http://id.loc.gov/authorities/genreForms/gf2011026703" | The URI that is expected to be in the results. |
| replacements | no value | Replacements are identified on following lines by indenting two spaces each replacement line. | ||
| maximumRecords | string | '20' | This is typically specified because the default is '4'. For accuracy tests, the value is usually set to '20' to test whether the expected value appears in the first 20 results. |
Example Tests:
Run a query for 'Casebooks'. If sub-authorities are supported by the authority, it will search all sub-authorities. It will look in the first 20 results for the subject_uri. If it is in the first 10 (by position value), the test passes. If it is in 11-20, the test gets a warning. If it is not in the first 20, the test fails.
-
query: Casebooks
position: 10
subject_uri: "http://id.loc.gov/authorities/genreForms/gf2011026115"
replacements:
maxRecords: '20'
Purpose: Run overtime to track the how quickly search results are returned by the system.
Used to Test Services:
The infrastructure for performance tests has not been implemented yet. Documentation of YML fields TBD.