US Core Testing Methodology

Inferno's US Core tests check to make sure servers are complying with the US Core R4 capability statement as described here. Each profile corresponds to its own sequence. Each sequence contains the following tests:

Authorization Test

This will send a search request for the resource associated with the sequence sans authorization token. We expect to receive either a 401 or 406 error as a response. The test fails if we receive anything else.

Search Tests

Each sequence contains one test for each 'SHALL' and 'SHOULD' search parameter as described by the capability statement.

Determining Search Parameters

The tests assume that we already have a patient selected from either the standalone or EHR launch. We will base our searches off of this patient (there are exceptions, but we will get to that later).

Most of the profiles have a search by patient. We will perform this search first using the patient ID we got earlier. We'll save the first resource returned from the response to base the rest of our searches. If no resources are found for the patient, then the rest of the sequence is skipped.

We look at the response from the first search to base all the subsequent searches. For example, the Allergy Intolerance sequence contains a search by patient + clinical-status. The patient parameter is the same patient id as the first search, and the clinical-status parameter is extracted by looking for AllergyIntolerance.clinicalStatus.coding.code where AllergyIntolerance is the resource returned from the first search.

Comparators

We also search for required comparators. Currently, the only comparators that are required are le, lt, ge, gt for certain date parameters. We extract the date from the first resource and then add or subtract a day depending on the comparator being used.

Search Validation

There are several components in validating the search results. The first is profile validation. We use FHIR Crucible to parse and validate the resource. This step will validate quantities and valuesets.

We also look through all the resources returned in the search and make sure they match the search parameters.

Interactions

We test each 'SHOULD' and 'SHALL' interaction (for example read, vread, history-instance).

Must Support Elements

Each profile contains some number of 'must support' elements. We follow the definition here to determine out testing procedure. Based on the wording of this guide, we decided to check if we've seen each of the must-support elements at least once from any of the returned resources. For example, we will check to see if any of the returned AllergyIntolerance resources in the AllergyIntolerance sequence contains a verificationStatus item.

Reference Resolution

We will verify that references to other resources will resolve. For example, if the AllergyIntolerance contains a Practitioner reference in its recorder item, then we will check if we can read Practitioner/id where id is specified by the reference.