ConceptScheme to define subsets of the activity list

[nickevansuk]

Proposer

EMD, imin, MyLocalPitch

Use Case

Applying simple rules to the master list allows it to be reduced in size for specific applications. This allows the master list to serve multiple usecases while its editors share a common understanding of likely patterns of use.

Original ConceptScheme:

- a
  - b
    - e
  - c
    - d
- f
- g
- h
- i
    
Example subset ConceptSchemes:

- a (rootConcept)
  - b
    - e
  - c
    - d

- a (rootConcept)
  - b (flatten)
  - e 
  - c (flatten)
  - d 

- a (flatten) (rootConcept)
- b (flatten)
- e 
- c (flatten)
- d 

- a (flatten) (rootConcept)
x b (flatten) (exclude)
- e 
- c (flatten)
- d 

- a (flatten) (rootConcept)
x b (exclude)
x e 
- c (flatten)
- d 

x a (flatten) (rootConcept) (exclude) - e.g. Group Exercise
x b (flatten) (exclude)
- e
- c (flatten)
- d 

Proposal

SKOS explicitly permits Concepts to be members more than one ConceptScheme: "One important feature of SKOS is that it is possible for the same concept to be linked to several concept schemes, using the skos:inScheme property." (ref)

In order to allow subsets of the master activity list to be published, this proposal suggests the use of ConceptScheme.

The arrays of hasTopConcept, inScheme, and topConceptOf within SKOS already permit the idea of a repurposed hierarchy.

Four specific rules are defined:

• rootConcept: Pins this Concept at the top level for the specific application, regardless of where it sits within the hierarchy of the parent ConceptScheme (master list). Note that narrower Concepts of this Concept might also be included in hasTopConcept if they are flattened, so rootConcept is distinct from hasTopConcept.
• flattenConcept (defined within the new ConceptSchemeRestriction): Links the narrower Concepts of a Concept to its broader Concept, so that they sit on the same level of the hierarchy as the its broader Concept (already supported by SKOS using broaderTransitive).
• excludeConcept (defined within the new ConceptSchemeRestriction): Excludes either only the specific Concept (if that Concept is also flattened), or the specific Concept and all narrower Concepts (if that Concept is not flattened). Exclude means not including the concept(s) in any user-facing hierarchy view or activity auto-complete, and not including Events related to the Concept in search results.
• hideConcept (defined within the new ConceptSchemeRestriction): Hides either only the specific Concept (if that Concept is also flattened), or the specific Concept and all narrower Concepts (if that Concept is not flattened). Hides means not displaying the concept(s) in any user-facing hierarchy view or activity auto-complete, but still including Events related to the Concept in search results.

Note that for a ConceptScheme no hierarchy is automatically inherrited from the parent ConceptScheme (master list) defined by the parentScheme, and that the hierarchy must be consistent with the constraints of the parentScheme. Hence within concept topConceptOf must be used to attach the top concepts to the ConceptScheme, and broaderTransitive must be used to define relationships within the ConceptScheme (consistent with the parentScheme).

Also note that excluded Concepts are not present in the new ConceptScheme, and users of the ConceptScheme are not expected to include any such excluded Concepts in their query results.

So if the parentScheme includes A -> B -> C:

• where C is excluded, a search for B will not return results containing C. C is not included in the ConceptScheme.
• where B is excluded but not flattened, a search for A will not return results containing B or C. B and C are not included in the ConceptScheme.
• where B is excluded and flattened, a search for A will not return results containing B, but will include results containing C. B is not included in the ConceptScheme.
• where B is flattened but not excluded, a search for B will still include results for C. A, B an C are included in the ConceptScheme.
• where B is hidden, a search for A will still include results for B and C, but the resulting breadcrums would exclude B. A, B an C are included in the ConceptScheme.

Hence exclude removes the Concept from use entirely. Whereas hide simply removes the Concept from any user-facing rendering of the ConceptScheme. For example, hiding has no effect on the results returned from a search.

New properties defined

• ConceptScheme: restriction of new type ConceptSchemeRestriction
• ConceptSchemeRestriction: parentScheme of type ConceptScheme
• ConceptSchemeRestriction: rootConcept
• ConceptSchemeRestriction: flattenConcept
• ConceptSchemeRestriction: excludeConcept
• ConceptSchemeRestriction: hideConcept
• Concept: hidden

Use in Opportunity API

Results for Events can contain activity including the new ConceptScheme as follows:

"activity" : {
  "type": "Concept",
  "id": "https://openactive.io/activity-list/#72d19892-5f55-4e9c-87b0-a5433baa49c8",
  "broaderTransitive" : { // flattened/excluded/hidden concepts in the chain are simply not included (even if they are topConceptOf) - this can be computed dynamically based on the restriction and the parentScheme
    "type": "Concept",
    "https://openactive.io/activity-list/#9a4375402-067d-4549-9d3a-8c1e998350a1"
    "broaderTransitive" : {
      "type": "Concept",
      "id": "https://openactive.io/activity-list/#9caeb442-2834-4859-b660-9172ed61ee71",
      "topConceptOf": [ "https://example.com/schemes/emd" ] //stop at topConceptOf which matches scheme
    }
  }
}

Filtering can be achieved via:

• activity.inScheme=https://example.com/schemes/emd (activities not included in the ConceptScheme are not returned in the results)
• activity=9caeb442-2834-4859-b660-9172ed61ee71 (for exact match)
• activity[include-narrower]=9caeb442-2834-4859-b660-9172ed61ee71 (for including all activities in the hieriarchy below the match)

Example

Using the following (abridged) master activity list:

The following defines a subset ConceptScheme:

{
  "@context": "https://www.openactive.io/ns/oa.jsonld",
  "id": "https://example.com/schemes/emd",
  "title": "Exercise, Move and Dance",
  "description": "The activity list for activities used by EMD UK",
  "type": "ConceptScheme",
  "license": "https://creativecommons.org/licenses/by/4.0/",
  "hasTopConcept": [ "https://openactive.io/activity-list/#984068a7-5b7b-4989-bb33-f96953d8960c", "https://openactive.io/activity-list/#d3b5104a-2e31-4cca-a278-ef8e0987a764" ],
  "restriction": {
    "type": "ConceptSchemeRestriction",
    "parentScheme": "https://openactive.io/activity-list/activity-list.jsonld",
    "rootConcept": [ "https://openactive.io/activity-list/#984068a7-5b7b-4989-bb33-f96953d8960c" ],
    "flattenConcept": [ "https://openactive.io/activity-list/#984068a7-5b7b-4989-bb33-f96953d8960c" ],
    "excludeConcept": [ "https://openactive.io/activity-list/#b976886d-d5f5-49c7-9502-008ab3d3d7a6" ],
    "hideConcept": [ "https://openactive.io/activity-list/#d3b5104a-2e31-4cca-a278-ef8e0987a764" ]
  },
  "concept": [
    {
      "id": "https://openactive.io/activity-list/#984068a7-5b7b-4989-bb33-f96953d8960c",
      "type": "Concept",
      "notation": "group_exercise",
      "prefLabel": "Group Exercise",
      "inScheme": [ "https://example.com/schemes/emd", "https://openactive.io/activity-list/activity-list.jsonld" ],
      "topConceptOf": [ "https://example.com/schemes/emd" ]
    },
    {
      "id": "https://openactive.io/activity-list/#d3b5104a-2e31-4cca-a278-ef8e0987a764",
      "type": "Concept",
      "notation": "dance_fitness",
      "prefLabel": "Dance Fitness",
      "hidden": true,
      "inScheme": [ "https://example.com/schemes/emd", "https://openactive.io/activity-list/activity-list.jsonld" ],
      "topConceptOf": [ "https://example.com/schemes/emd" ]
    },
    {
      "id": "https://openactive.io/activity-list/#20532927-bb47-4bdf-b341-4966336baf11",
      "type": "Concept",
      "notation": "azonto_fitness",
      "prefLabel": "Azonto Fitness",
      "inScheme": [ "https://example.com/schemes/emd", "https://openactive.io/activity-list/activity-list.jsonld" ],
      "broaderTransitive": [ "https://openactive.io/activity-list/#d3b5104a-2e31-4cca-a278-ef8e0987a764" ]
    }
  ]
}

Questions

• Do we need ConceptSchemeRestriction or can flattenConcept, excludeConcept and hideConcept be defined on ConceptScheme?
• Is parentScheme the right name for that property?
• Should we define restriction/ConceptSchemeRestriction at all at this stage, or just extension props for that for now and just recognise broaderTransitive (array), inScheme (array), and hasTopConcept (array) in the modelling spec for now, along with the use of ConceptScheme for this purpose?

[lukehesluke]

@nickevansuk just want to make sure I understand this first. Please correct me if I'm wrong about any of these points

• In the example you gave, a new ConceptScheme has been created, with id https://example.com/schemes/emd
• restriction.parentScheme: this ConceptScheme "inherits" from https://openactive.io/activity-list/activity-list.jsonld (the "master list")
• restriction.rootConcept: this ConceptScheme's top concept (i.e. the top of the hierarchy) is Group Exercise
• restriction.flatten: child concepts of "Group Exercise" will be flattened out into the list. In this example "Dance Fitness". This means that this ConceptScheme's concepts will now be: ["Group Exercise", "Dance Fitness","Azonto Fitness"], but that "Dance Fitness" becomes a topConcept (as it has been flattened to the top level) in the "concept" list.
• restriction.exclude: This ID refers to "AzontoBeats", which is excluded from the ConceptScheme.
• restriction.hide: "Dance Fitness" will be hidden as a breadcrumb/facet from search results, but events with this activity will still be shown

At this point we have a clearly defined ConceptScheme. It inherits from the master list but only includes the results of "Group Exercise" and under. These are flattened, with one excluded and one hidden. This looks like a complete description of a collection of Concepts

So why then is there also a concept field with a list of concepts? If this doesn't exactly match what has been defined in the above fields, what is the ConceptScheme

In the example above, Concept "Azonto Fitness" is included with field broaderTransitive set to the "Dance Fitness" Concept. Does this mean that this is an extra Concept that is tacked on to the scheme at the same hierarchical level as "Group Exercise"?

If the above is true, then this is clearly a necessary extra piece of information, but why then is the "Dance Fitness" concept redefined this array given as it is already implicitly mentioned in the ConceptScheme's root hasTopConcept field?

[nickevansuk]

@lukehesluke sorry the example IDs weren't accurate at all so I've updated them to make the example based on a real list, and I've edited your comment to match the updated example - so please read through and see if the questions you had still make sense / are needed?

[jury89]

Hi @nickevansuk
the structure you proposed works really great for us.
However, trying to validate your example, i get this 3 errors:

• url: Required field "url" is missing from "ConceptScheme".
• @context: The @context field is required in the root node of your data. It should contain the OpenActive context (https://www.openactive.io/ns/oa.jsonld) as a string or the first element in an array.
• inScheme: Invalid type, expected "http://schema.org/url" but found "ArrayOf#http://schema.org/url".

We could replace id in ConceptScheme with url and context with @context.
I'm not sure instead which is the best solution for the last error with the attribute inScheme. Perhaps changing the validator and allowing ArrayOf#http://schema.org/url could be a solution.

[jury89]

We are working on an API serving Football events, so we created a subset including Football, 5-a-side, 6-a-side, 7-a-side, 8-a-side, 9-a-side, 11-a-side.
In order to do so we did:

• flatten: Small Sided Football
• exclude: Blind Football, Futsal, Small Sided Football, Walking Football, Powerchair Football, Wheelchair Football
• rootConcept: Football

This is how the response looks like:

{
  "@context": "https://www.openactive.io/ns/oa.jsonld",
  "@type": "ConceptScheme",
  "id": "http://my-url.com/activity-list",
  "title": "My Local Pitch Activity List",
  "description": "List of activities available for MLP events",
  "ext:restriction": {
    "@type": "ConceptSchemeRestriction",
    "parentScheme": "https://openactive.io/activity-list/activity-list.jsonld",
    "rootConcept": [
      "https://www.openactive.io/activity-list/#0a5f732d-e806-4e51-ad40-0a7de0239c8c"
    ],
    "exclude": [
      "https://www.openactive.io/activity-list/#6016ce87-d9ed-4bd6-8cc9-5598c2f59f79",
      "https://www.openactive.io/activity-list/#22fe3033-b0e4-4717-8455-599180b5bcba",
      "https://www.openactive.io/activity-list/#1de4c90e-6a27-4bc4-a2be-437a443c7ded",
      "https://www.openactive.io/activity-list/#b8019b67-2ade-406f-a012-91a5c3869652",
      "https://www.openactive.io/activity-list/#f6301564-93d5-41ff-91a1-7ac2dd833951",
      "https://www.openactive.io/activity-list/#666cf454-4733-4697-89cb-8e28f6e8595b"
    ],
    "flatten": ["https://www.openactive.io/activity-list/#22fe3033-b0e4-4717-8455-599180b5bcba"]
  },
  "concept": [
    {
      "@type": "Concept",
      "id": "https://www.openactive.io/activity-list/#0a5f732d-e806-4e51-ad40-0a7de0239c8c",
      "notation": "football",
      "prefLabel": "Football",
      "inScheme": [
        "http://my-url.com/activity-list",
        "https://openactive.io/activity-list/activity-list.jsonld"
      ],
      "definition": "Football is widely considered to be the most popular sport in the world. The beautiful game is England's national sport",
      "topConceptOf": "http://my-url.com/activity-list"
    },
    {
      "@type": "Concept",
      "id": "https://openactive.io/activity-list/#117e7f70-6c42-4b1f-a3bb-620b63ea2632",
      "notation": "11_a_side",
      "prefLabel": "11-a-side",
      "inScheme": [
        "http://my-url.com/activity-list",
        "https://openactive.io/activity-list/activity-list.jsonld"
      ],
      "broaderTransitive": "https://www.openactive.io/activity-list/#0a5f732d-e806-4e51-ad40-0a7de0239c8c"
    },
    {
      "@type": "Concept",
      "id": "https://openactive.io/activity-list/#2c21b96d-1286-4e17-9079-d268c0a276d5",
      "notation": "9_a_side",
      "prefLabel": "9-a-side",
      "inScheme": [
        "http://my-url.com/activity-list",
        "https://openactive.io/activity-list/activity-list.jsonld"
      ],
      "broaderTransitive": "https://www.openactive.io/activity-list/#0a5f732d-e806-4e51-ad40-0a7de0239c8c"
    },
    {
      "@type": "Concept",
      "id": "https://openactive.io/activity-list/#19b36744-bbcd-4999-8c0d-0a7985dc475e",
      "notation": "8_a_side",
      "prefLabel": "8-a-side",
      "inScheme": [
        "http://my-url.com/activity-list",
        "https://openactive.io/activity-list/activity-list.jsonld"
      ],
      "broaderTransitive": "https://www.openactive.io/activity-list/#0a5f732d-e806-4e51-ad40-0a7de0239c8c"
    },
    {
      "@type": "Concept",
      "id": "https://openactive.io/activity-list/#55e525a5-615a-4c3f-a889-9049c95c94cf",
      "notation": "7_a_side",
      "prefLabel": "7-a-side",
      "inScheme": [
        "http://my-url.com/activity-list",
        "https://openactive.io/activity-list/activity-list.jsonld"
      ],
      "broaderTransitive": "https://www.openactive.io/activity-list/#0a5f732d-e806-4e51-ad40-0a7de0239c8c"
    },
    {
      "@type": "Concept",
      "id": "https://openactive.io/activity-list/#f816f3b5-3128-4421-b71a-25cc7c1e1880",
      "notation": "6_a_side",
      "prefLabel": "6-a-side",
      "inScheme": [
        "http://my-url.com/activity-list",
        "https://openactive.io/activity-list/activity-list.jsonld"
      ],
      "broaderTransitive": "https://www.openactive.io/activity-list/#0a5f732d-e806-4e51-ad40-0a7de0239c8c"
    },
    {
      "@type": "Concept",
      "id": "https://openactive.io/activity-list/#64ba748e-d7f0-46bd-ac49-63820c5eb10e",
      "notation": "5_a_side",
      "prefLabel": "5-a-side",
      "inScheme": [
        "http://my-url.com/activity-list",
        "https://openactive.io/activity-list/activity-list.jsonld"
      ],
      "broaderTransitive": "https://www.openactive.io/activity-list/#0a5f732d-e806-4e51-ad40-0a7de0239c8c"
    }
  ]
}

[nickevansuk]

Just saw your comment @jury89, great points:

We could replace id in ConceptScheme with url and context with @context

• this is a typo, good catch, have updated the above!

I'm not sure instead which is the best solution for the last error with the attribute inScheme Perhaps changing the validator and allowing ArrayOf#http://schema.org/url could be a solution.

• agree with this, suggest we update this in the spec (@ldodds have raised this as a comment in the V2 spreadsheet - https://docs.google.com/spreadsheets/d/1jpw-cSdKygBF5UXIAYvrurkXRe8mu-sPClVb5yrMtPc/edit)

url: Required field "url" is missing from "ConceptScheme".

• Not sure why this is required, as we already have id required, also queried for V2 in the same spreadsheet.

@petewalker FYI ^ validator in real use!

[lukehesluke]

@nickevansuk thanks for updating everything. The questions I still have are:

• what is broaderTransitive?
• If the ConceptScheme as a collection of Concepts is entirely defined by the hasTopConcept and restriction fields, why is there a concept field which lists all the concepts that are already implicitly defined by the former fields?

[nickevansuk]

Sure no problem:

what is broaderTransitive?

broaderTransitive is defined in SKOS, and is used to draw inferences when a part of the hierarchy might be flattened.

So for this assertion:

• Group Exercise -broader-> Fitness -broader-> Zumba -broader-> Zumba Gold

The following are true:

• Group Exercise -broaderTransitive-> Zumba Gold
• Group Exercise -broaderTransitive-> Zumba -broaderTransitive-> Zumba Gold
• Group Exercise -broaderTransitive-> Fitness -broaderTransitive-> Zumba -broaderTransitive-> Zumba Gold

Because broaderTransitive is a superset of the broader relation, the proposal is using broaderTransitive to render the result of the restricted hierarchy for consistency (rather than having a mix of broader and broaderTransitive depending on the flattening).

There's some great diagrams on this here: https://www.w3.org/TR/skos-primer/#sectransitivebroader

If the ConceptScheme as a collection of Concepts is entirely defined by the hasTopConcept and restriction fields, why is there a concept field which lists all the concepts that are already implicitly defined by the former fields?

The concept property here is optional and for convenience, if say the data user didn't want to apply the rules to render the new list of concepts, the provider could render it for them into concept. You're right, and in fact it's entirely defined by just the restriction fields.

[nickevansuk]

Additional suggestion based on a conversation with @domfennell:

• ConceptSchemeRestriction should include a maximumVisibleDepth which is an integer that indicates how many levels deep the hierarchy should be rendered. When the restricted list has been generated using the steps outlined above, the maximumVisibleDepth is applied as a final step, which effectively marks the next level of Concepts below that depth as "hidden": true (which hence applies to all their children). This reduces the need to have a very large array of hideConcept maintained the restriction administrator as the activity list evolves.

• Clarification point (as per @lukehesluke's comment above) that hideConcept may or may not affect the search results, that's an implementation detail for the data consumer (and perhaps should be configurable?)