Proposal: Consistent levels

[nickevansuk]

Proposer

imin and EMD

Use Case

When presenting data from multiple sources, inconsistent levels become a problem for both display and filtering.

Why is this not covered by existing properties?

The existing model allows free-form levels, which does not allow for easy categorisation.

Please provide a link to example data

EMD UK, Classfinder, Our Parks, all have the notion of a "beginner", and most have "intermediate" and "advanced" too (Our Parks uses "Moderate" and "Hard").

England Netball has "beginner".

SportStarta have published their level data: http://www.devapi.lewishamparklife.co.uk/api/concepts/level

British Cycling uses "Easygoing", "Steady" and "Challenging".

ClubSpark has "All" for many sessions.

Requirements

The priority is to agree a standard way to represent a beginner or entry level session (which is one of the main objectives of OpenActive). It would also appear that language around some notion of both "intermediate" and "advanced" is common. Expressing mixed ability / "open to all" would also be useful.

Similar to amenityFeature it is useful to allow a custom label ("Easygoing") as well as a defined term (Beginner).

It is also useful to mark sessions as catering for multiple levels e.g. '"Beginner and Intermediate"

Example

As a starter for 10:

  "level": [
     {
        "type": "OpportunityLevelBeginner",
        "name": "Easygoing"
     },
     {
        "type": "OpportunityLevelIntermediate",
        "name": "Moderate"
     },
     {
        "type": "OpportunityLevelAdvanced",
        "name": "Hard"
     }
  ]

And:

  "level": [
     {
        "type": "OpportunityLevelMixed",
        "name": "Open To All"
     }
  ]

[peter-dolkens]

Would a level rating (0-100) perhaps work better here?

So internally the systems might rank "EasyGoing" as a 0-30, "Moderate" as a 30-70+, and "Hard" as a 70+

This would allow external consumers with more, or less levels, to set their own boundaries against a scale of 0-100.

0-50 / 50-100 split for Easy/Hard - or more granular splits if you offer more elite courses perhaps?.

This is similar to the way font-weights work (see here for reference) in that there are "well known" values (400 for normal, 700 for bold), but then if the systems (fonts) support it, you can get lighter and heavier weights too, with reliable fall-backs.

[nickevansuk]

Another example of multiple levels from the EMD UK data, this class is suitable for beginners, intermediate and advanced:

"level": [
  "Beginners",
  "Intermediate",
  "Advanced"
]

[nickevansuk]

Further thoughts:

On the basis that every source seems to have some notion of easy, medium, hard: define an approach where the names can be customised for specific types of activities or data sources, within standard types for "Beginner", "Intermediate", and "Advanced". Data users can then choose to display consistent labels or the provider's own labels.

We could also include a difficulty value on each for the case where multiple are supplied, as @dolkensp suggests. This is similar to defining "Bold" as 700.

To allow for the simplest implementation of this:

• OpportunityLevel is the parent type, which should not be used directly
• OpportunityLevelBeginner defaults difficulty to 200, and name to Beginner
• OpportunityLevelIntermediate defaults difficulty to 500, and name to Intermediate
• OpportunityLevelAdvanced defaults difficulty to 800, and name to Advanced

This gives room for 3 levels within each level, if required.

This also allows for transparency what will likely appear under what filter in a user's experience e.g. there might be 8 levels of a martial art, where 5 are considered advanced, and helps with validation.

So suggested mapping to fonts:

Value	Font	OpportunityLevel
100	Thin (Hairline)
200	Extra Light	OpportunityLevelBeginner
300	Light
400	Normal
500	Medium	OpportunityLevelIntermediate
600	Semi Bold (Demi Bold)
700	Bold
800	Extra Bold (Ultra Bold)	OpportunityLevelAdvanced
900	Black (Heavy)

Additionally in terms of validation (providing a range of 300 on each):

• OpportunityLevelBeginner must have difficulty in range 50 to 350
• OpportunityLevelIntermediate must have difficulty in range 350 to 650
• OpportunityLevelAdvanced must have difficulty in range 650 to 950

The other nice property of this 50-950 range is that you can create a scale of 1-10 difficulty score out of it quite easily with Math.round(difficulty/100).

So simple case:

  "level": [
     {
        "type": "OpportunityLevelBeginner"
     },
     {
        "type": "OpportunityLevelIntermediate"
     },
     {
        "type": "OpportunityLevelAdvanced"
     }
  ]

Simple case with custom labels:

  "level": [
     {
        "type": "OpportunityLevelBeginner",
        "name": "Easygoing"
     },
     {
        "type": "OpportunityLevelIntermediate",
        "name": "Moderate"
     },
     {
        "type": "OpportunityLevelAdvanced",
        "name": "Hard"
     }
  ]

Custom labels and more than 3 levels:

  "level": [
     {
        "type": "OpportunityLevelBeginner",
        "name": "Easygoing"
     },
     {
        "type": "OpportunityLevelIntermediate",
        "name": "Light work",
        "difficulty": 400
     },
     {
        "type": "OpportunityLevelIntermediate",
        "name": "Sweating somewhat",
        "difficulty": 600
     },
     {
        "type": "OpportunityLevelAdvanced",
        "name": "Hard"
     }
  ]

What do you think @ldodds? Given most sources have 3 levels are we overcomplicating this, and should we have them use the names alone to differentiate between levels where there are more than 3? But does that limit potential uses of the data?

I guess the aim here is to create uniformity across disparate data sources for simple aggregation use cases without losing potentially useful granularity for more complex / interesting use cases?

[nickevansuk]

To add a comment from Try Tag rugby, there's also an additional level of "no experience" that's a useful classification, as "Beginner" assumes they already understand the rules and have played a few times.

[thill-odi]

I'm wondering about the extent to which any difficulty level needs to be extensible. Off the top of my head/doing some quick Googling, it doesn't take long to start hitting edge cases - which makes me think they're not really 'edge cases' at all.

For example:

• climbing sports often have a distinct category for 'technical' climbs: the climb is not just physically challenging but also requires skill and ingenuity to achieve, and/or the use of specialised equipment.
• many martial arts have a well-defined 'belt' or kyu/dan system: https://en.wikipedia.org/wiki/Dan_(rank)#Usage_in_martial_arts
• this yoga studio has six grades defined: https://iyengaryogaencinitas.com/class-info

One way to deal with this would be to view difficulty rating as a hierarchy, with 'Easy', 'Intermediate' and 'Difficult' as top-level terms. As a data-publisher, I would want to be able to use my own terms and either map them or subordinate them to these broad catch-all terms; as a data-consumer, I might also want to be able to see a definition of these terms, how standardised they are, and/or a URL to where they are defined.

For example, for the sport of Aikido:

Intermediate has narrowerTerm 3Kyu, with definition "have at least 50 days experience", asRecognisedBy "The International Aikido Federation" and described in more detail at http://www.aikikai.or.jp/eng/information/review.html.

For snorkelling:

Easy has narrowerTerm Dolphin with definition "Has completed BSAC Dolphin certificate' as recognised by Organisation "The British Sub-Aqua Club' and described in more detail at 'https://www.bsac.com/training/snorkelling-courses/'

[thill-odi]

Another open question is how we relate this to age-ranges: it's often the case (as in the snorkelling example above) that there's a distinct category of skill-levels and certifications for children. We'll need to think about how we capture these cross-cutting concerns.

[ldodds]

Reviewing the initial proposal again, it seems like there's two broad use cases:

• A. individual sports and activities have their own rankings/experience/difficulty levels. some sessions are designed for those abilities, so its important to be able to communicate
• B. some sessions are more open, e.g. to beginners, having a standard way to tag these is useful to help reach the audience that OpenActive is interested in

I think A requires us to:

• have a way to indicate the difficulty level, we have that with oa:level
• have a means to publish and share small controlled vocabularies for specific sports. we can do that using SKOS, but need to be able to publish the lists somewhere. If concepts have descriptions, then for display purposes, users can be given more information

There's also an argument to be made for having a simple beginner/intermediate/advanced set of levels for people that have very simplistic classifications. But at @thill-odi its not clear how widely used that is. I'd recommend exploring usage of the oa:level property and identifying commonalities in the published data.

To handle B, it may be that a standard set of levels will help meet this requirement. Although I note in the comments that we have several distinctions:

• "no experience required" - for people who have not participated in this type of event before
• "beginners" - which might imply no experience required, but in some contexts might mean "people who have an understanding of the rules"
• "open for all" - a session which is open for mixed ability levels
• "experienced" - for people with some experience

So I'm wondering if instead we defined these terms as a controlled vocabulary and recommend that they are used for the oa:level property, then we can avoid getting into a more complex discussion around aligning vocabulary levels. These categories should focus on our core use case, while deferring more complex categories to their expert communities.

The recommendation would be that Events should have at least one value for oa:level which draws from the above category list. For those that are for Experienced people, then there should be another level that provided more information using a term used from a more specific vocab

E.g

{
 "type": "Session",
 "level": [{
    "id": "",
    "type": "Concept",
    "prefLabel": "Beginner",
    "inScheme": "http://openactive.io/levels/standard"
  }]
}

{
 "type": "Session",
 "level": [{
    "id": "",
    "type": "Concept",
    "prefLabel": "Experienced",
    "inScheme": "http://openactive.io/levels/standard"
  },{
    "id": "",
    "type": "Concept",
    "prefLabel": "3Kyu",
    "inScheme": "http://example.org/aikkido/levels"
  }
]
}

[thill-odi]

Reviewing the thread so far:

Given that some set of broad terms used across all activities is evidently a useful thing to have, and that more specialised terms are also useful and frequently encountered, much of the above discussion is substantially about how we map these more specialised terms to the general terms. Broadly speaking, the approaches suggested fall into two camps:

1. Use of a numerical scale, along the lines of font-weight
2. A SKOS-y approach, where the mapping is semantic and between terms

So, for the sake of example, the first might look something like this:

{ "type": "Session", "level": [{ "name": "Orange Belt", "numericRanking": 400, "description": "The third level of accomplishment, above White and Yellow belts" }] }

The second might be:

{ "type": "Session", "level": [{ "id": "http://openactive.io/levels/standard/experienced", "type": "Concept", "prefLabel": "Experienced", "inScheme": "http://openactive.io/levels/standard" },{ "id": "http://example.org/aikkido/levels/orange", "type": "Concept", "prefLabel": "Orange Belt", "inScheme": "http://example.org/aikkido/levels", "definition": ""The third level of accomplishment, above White and Yellow belts", "broadMatch": "http://openactive.io/levels/standard/experienced" } ] }

(@ldodds might spot that this is basically what he wrote, but that the implicit equivalence in the examples he gave above is now explicit).

The advantages of the former approach are that:

1. It is inherently ordered
2. It leaves the question of banding and terminology to the client
3. Assuming the numeric range is somewhere on the order of 0-1000, there's plenty of room for fine-grained distinctions
4. It's potentially much more terse than the SKOS approach.

The advantages of the latter are:

1. It's easier to capture distinctions such as 'Open to All' or 'No experience required' than with the numeric scale
2. It supports decentralised data-publishing more readily than a numeric scale
3. The rationale behind mappings is more transparent and self-contained than the justification for numeric assignments
4. It's arguably more intuitive

Does this sound like a fair summary of the thread and the issues it raises so far?

[ldodds]

Couple of additional comments:

• for consistency in approach I think we should be basing this around SKOS and concepts, rather than coming up with a new way to describe categories which the current numeric ranking based approach is leaning towards
• I'm not keen on the numeric ranking style as creates accidental precision, or an expectation that rankings might be comparable when they're not. I think really the intention is just to capture an ordering
• an ordering can be added to the SKOS style approach as illustrated in previous examples by adding a sorting property to concepts, defined as providing a suggested ranking order for all concepts in that scheme. It would be similar to CSS z-index and define a relative order

The other thing that I think is missed in your summary @thill-odi is that I was proposing:

• encouraging orgs to publish their own vocabularies to deal with bespoke use cases, as a way to support them but without having to coordinate across community
• recommending SKOS because it does provide support for defining equivalences if that's helpful
• focusing our work on defining the different categories of beginner/no skills required, as its more helpful for OA use cases. What I've suggested is partitioning the space to separate out classes for more experienced people and instead focusing on the nuances for beginners.

[thill-odi]

Reviewing and collating the evidence above, in the prerequisites issue, as well as some of the feedback from the questionnaire, I come up with this list:

• Aikido
• Archery
• British Cycling
• Classfinder
• Clubspark
• Cycling
• EMD UK
• England Netball
• Group exercise
• Health Commitment Statements: background and example
• Hockey
• Iyengar Yoga
• Our Parks
• Snorkelling
• SportStarta
• Stand alone Gyms, leisure centres, sports centres, national sports centres.
• Swimming
• Swim Grades
• Tennis
• Triathlon

Reviewing this, and surveying the population of oa.level in the endpoint analyzer, it seems that the second of @ldodds proposals covers the range of level-description required. It may be advisable, however, to make skos:definition a RECOMMENDED property of oa:level.

[nickevansuk]

@thill-odi as the thread above seems to cover quite a bit of ground, would it be possible to summarise the proposal that your analysis indicates is the best way forward? Including a couple of examples? This will make it easier to review / discuss?

[thill-odi]

The proposal would be, first, to define a small controlled vocabulary consisting of the terms NoExperienceRequired. Beginner, Open for All, and Experienced, with the option of linking to a more fine-grained scheme.

Like so:

{
 "type": "Event",
 "level": [{
    "id": "https://openactive.io/levels/standard/beginner",
    "type": "Concept",
    "prefLabel": "Beginner",
    "inScheme": "http://openactive.io/levels/standard"
  }]
}
{
 "type": "Event",
 "level": [{
    "id": "https://openactive.io/levels/standard/experienced",
    "type": "Concept",
    "prefLabel": "Experienced",
    "inScheme": "http://openactive.io/levels/standard"
  },{
    "id": "https://example.org/aikido/levels/3kyu",
    "type": "Concept",
    "prefLabel": "3Kyu",
    "inScheme": "http://example.org/aikido/levels",
    "definition": "4Kyu + 100 Training Days" 
  }
]
}

[nickevansuk]

Sure - So for example, from EMD UK’s perspective: “Intermediate” and “Advanced” would both be narrower terms of “Experienced”? Is it worth including these in the official OA list as they are so common?

Ok great, and I guess we’ll need to clearly define “No Experience Required”, “Beginner” and “Open for All”?

Does the “Open for All” distinction basically mean “we don’t have levels for this activity”?

[thill-odi]

Actions from the W3C call: add 'Intermediate' to the term list, and change 'Experienced' to 'Advanced'. Otherwise the proposal stands.

[tommarley148]

The proposal would be, first, to define a small controlled vocabulary consisting of the terms NoExperienceRequired. Beginner, Open for All, and Experienced, with the option of linking to a more fine-grained scheme.

Like so:

{
 "type": "Event",
 "level": [{
    "id": "https://openactive.io/levels/standard/beginner",
    "type": "Concept",
    "prefLabel": "Beginner",
    "inScheme": "http://openactive.io/levels/standard"
  }]
}
{
 "type": "Event",
 "level": [{
    "id": "https://openactive.io/levels/standard/experienced",
    "type": "Concept",
    "prefLabel": "Experienced",
    "inScheme": "http://openactive.io/levels/standard"
  },{
    "id": "https://example.org/aikido/levels/3kyu",
    "type": "Concept",
    "prefLabel": "3Kyu",
    "inScheme": "http://example.org/aikido/levels",
    "definition": "4Kyu + 100 Training Days" 
  }
]
}

Happy with this as discussed on previous w3c call - we are being asked for this feature within our products by a few customers and would make it easier for people to find the suitable activity for their level so we definitely support this alongside our customers and the spec proposed.

[nickevansuk]

Relevant points from the summary of the W3C call on 25 March 2020:

• Standardise only the string “Beginner” for now, as a short term solution to save trying to solve the larger levels problem for the COVID-19 / Virtual Event usecase