Introduce TestDataRequirements in order to ensure Booking Systems have exactly the right data for running tests

[lukehesluke]

User stories

"As a Booking System, I would like to know what opportunity data I need to have in my platform in order to run the test suite (for my selection of implemented features) in random mode."

"As a Booking System, I would like to have, for each criteria, a detailed set of instructions that I can use to either (1). Create an opportunity matching that criteria in controlled mode (2). Ensure that I have an opportunity matching that criteria in random mode."

Proposed Solution

Specify Opportunity and Offer requirements using Shape Expressions, described in this comment: #1 (comment)

Proposed Solution (Obsolete)

TestDataRequirements is an attempt to solve the 2 user stories above. This has been partially implemented in this PR: openactive/openactive-test-suite#326.

An OpportunityProfile using TestDataRequirements looks like:

{
  "@type": "ScheduledSession",
  "superEvent": {
    "@type": "SessionSeries",
    "organizer": {
      "@type": "Organization",
      "@id": "https://id.booking-system.example.com/organizer/3"
    }
  },
  "test:testOpportunityCriteria": "https://openactive.io/test-interface#TestOpportunityBookable",
  "test:testOpportunityDataRequirements": {
    "@type": "test:OpportunityTestDataRequirements",
    "test:startDate": {
      "@type": "test:DateRange",
      "minDate": "2020-12-20",
      "maxDate": "2020-12-25",
    },
    "test:remainingCapacity": {
      "@type": "QuantitativeValue",
      "minValue": 1,
    }
  },
  "test:testOfferDataRequirements": {
    "@type": "test:OfferTestDataRequirements",
    "test:price": {
      "@type": "QuantitativeValue",
      "minValue": 0.01,
    },
    "test:prepayment": {
      "@type": "test:OptionRequirements",
      "valueType": "oa:RequiredStatusType",
      "blocklist": ["https://openactive.io/Unavailable"]
    }
  }
}

Notes on the above example

• test:testOpportunityDataRequirements includes requirements which are only related to the opportunity and not the offer

  test:testOfferDataRequirements includes requirements which are related to the offer (some of these involve both the offer and the opportunity)

• Each FieldIdentifier (e.g. test:startDate) refers to a field in the Opportunity/Offer. It's value can be any of the accepted SingleFieldRequirementTypes e.g. test:DateRange, QuantitativeValue, etc.

Controlled Mode

For controlled mode, a Booking System, would receive this info via the Test Interface endpoint: POST /test-interface/datasets/:testDatasetIdentifier/opportunities (link).

The Booking System could then use the requirements to generate the Opportunity

Random Mode

For random mode, a document could be generated by Test Suite which includes a list of all Opportunity types that will be needed and in what quantities.

This could look like:

{
  "@context": [
    "https://openactive.io/",
    "https://openactive.io/test-interface"
  ],
  "@type": "ItemList",
  "numberOfItems": 123,
  "itemListElement": [
    {
      "@type": "ListItem",
      "item": {
        "@type": "ScheduledSession",
        // ... Test Data Requirements. As in the example at the top of this issue
      },
      "test:numberOfInstancesInDistribution": 32
    },
    // ... more ListItems
  ]
}

A Booking System would then use this document to prepare for a Test Suite run by either (1). Seeding some test data in its database or (2). Ensuring that its database already has test data that matches all of the requirements.

Definition

In TestDataRequirements.d.ts, you can see, in TypeScript, the full definition for TestDataRequirements that is sufficient to fully describe all of the criteria enumerated in the Test Interface.

Questions

We'd like to get feedback on the approach in general

However, one particular issue is that the FieldIdentifiers will clash with existing schema.org fields (e.g. price / https://schema.org/price, startDate / https://schema.org/startDate). The semantic meaning here is different from that in schema.org. As an example, price here will refer to a requirement on price (e.g. "max price of £10") rather than a specific price.

Therefore, we need to find some way of distinguishing these fields from those already in schema.org.

Some options:

• Use a common prefix/suffix e.g. *Requirement. price -> priceRequirement.


  This generally ends up with clashes e.g. https://schema.org/advanceBookingRequirement, will clash with our requirement field for advanceBooking


  *Specification has the same issue as there is already an existing https://schema.org/priceSpecification

• Use a hyphenated suffix like in schema.org actions uses for <property>-input / <property>-output.



  price -> price-requirement



  We can document TestDataRequirements similarly to actions by stating that, for each <property>-requirement, the expected type may only be one of the allowed SingleFieldRequirementTypes (e.g. test:StartDate, QuantitativeValue, etc).

  Our usage would differ from schema.org's usage for actions, however. This is because, while, for actions, you can use any property identifier before -input or -output (e.g. ffffjskdlfj-input is acceptable), in our use case there would be an explicitly defined set of legal fields i.e. price-requirement, remainingCapacity-requirement, etc.

[nickevansuk]

Following a suggestion from @ldodds, perhaps we can borrow from Shape Expressions here (https://shex.io/shex-semantics/index.html)?

For example:

    {
      "@type": "ListItem",
      "test:numberOfInstancesInDistribution": 8,
      "item": {
        "@type": "Slot",
        "facilityUse": {
          "@type": "FacilityUse",
          "provider": {
            "@type": "Organization",
            "@id": "https://localhost:5001/api/identifiers/sellers/1"
          }
        },
        "test:testOpportunityCriteria": "https://openactive.io/test-interface#TestOpportunityBookableNonFreePrepaymentRequired",
        "test:testOpportunityDataShapeExpression": [
          {
            "@type": "test:TripleConstraint",
            "predicate": "https://openactive.io/remainingUses",
            "valueExpr": {
              "@type": "NumericNodeConstraint",
              "mininclusive": 2
            }
          }
        ],
        "test:testOfferDataShapeExpression": [
          {
            "@type": "test:TripleConstraint",
            "predicate": "https://schema.org/price",
            "valueExpr": {
              "@type": "NumericNodeConstraint",
              "mininclusive": 0.01
            }
          },
          {
            "@type": "test:TripleConstraint",
            "predicate": "https://openactive.io/prepayment",
            "valueExpr": {
              "@type": "test:OptionNodeConstraint",
              "datatype": "oa:RequiredStatusType",
              "allowlist": [
                "https://openactive.io/Required"
              ]
            }
          }
        ]
      }
    }

See full example below:

test-data.txt

It simplifies from a modelling perspective, as we don’t need to redefine a "constraint/requirements" version of each property… but does that come at the cost of readability?

[lukehesluke]

@nickevansuk the example you've given looks pretty readable. It looks like we'll be using some of our defined data types anyway like OptionConstraint, ArrayConstraint, etc.

I can definitely see the benefit of using an existing spec. For example, we and others could use ShEx tools. Maybe we could use https://github.com/shexSpec/shex.js/ for validating data from Booking Systems (as opposed to the current manual validation logic).

I've only given it a cursory look so far. Have you come across any potential downsides to using it in what you've looked at so far?