Referencing multiple pieces of evidence and related metadata in Assertions

[timothyfcook]

Allow multiple entries in an evidence array and add a descriptive vocabulary to enable embedding evidence metadata directly in an Assertion

1. As a badge issuer, I want to include some metadata about the evidence that demonstrates a recipient's achievement of the badge, so that a representation of this evidence can be provided to viewers wherever the badge is displayed.
2. As a badge recipient, I want my assertion to contain and fully describe all the pieces of evidence that are relevant to my badge so that I can display the entirety of the experience behind the badge to consumers.

[kayaelle]

May also wish to add (but may be out of scope): add privacy protection to each piece of evidence. For instance if the evidence should be private and accessible only to the recipient and issuer. Could encrypt individual pieces of evidence.

[ottonomy]

Just a note for readers, the ability to add multiple pieces of evidence is a feature of the Linked Data model. The 2.0 proposal will allow evidence to be a field that accepts multiple values in the Assertion class.

Acceptable options:
Single IRI value:

{ "evidence": "http://example.com/portfolio-1" }

Multiple IRI values:

{ "evidence": ["http://example.com/portfolio-1", "http://example.com/portfolio-2"] }

Rich metadata value:

{
  "evidence": [
    "http://example.com/portfolio-1" , 
    {
      "id": "http://example.com/portfolio-2",
      "name": "My poem",
      "description": "A short poem about a fuzzy dog",
      "schema:genre": "Poetry",
      "schema:audience": "Children, ages 3-6"
    }
  ]
}

Vocabulary elements to be included, mandatory/optional requirements TBD.

[szerge]

I see two options:

1. we keep a single (link to a) piece of evidence (e.g. a portfolio) that contains itself multiple pieces of evidence; or
2. we have (links to) multiple pieces of evidence within the badge itself, therefore we need to add some form of narrative to make sense of those different pieces — the narrative is sometimes the most important element in an evidence.

Option 1 keeps badges simple but assumes that earners have access to some kind of ePortfolio (so we assume the existence of a more 'complete' infrastructure'), option 2 brings the badge closer to a micro-portfolio without having to expect the existence of an external ePortfolio — and thus opening the door to reinventing ePortfolio systems...

My personal choice would be for option 2:

1. the narrative is a critical part of most evidence — an artefact doesn't always speak by itself;
2. a badge reader can access the narrative without any additional click on the badge;
3. no need for a ePortfolio platform.

The structure for evidence field could like like this:

• 'narrative' built around pieces of evidence
• one or more 'links' to artefacts used as supporting evidence

JSON:

{
"evidence": { 
    "narrative": "I have contributed to the 2.0 Open Badge specification by regularly contributing to
    the ba-standards group call, c.f. [1] and [2], and making written suggestions, c.f. [3]. 
    My goal was to make badges the building blocks of a different kind of ePortfolio, more open,
    modular and flexible ..."
    "links": [{
        "id": "http://etherpad.badgealliance.org/ba-standard-2016-11-10",
        "evidenceIndex": "1",
        "criteriaIndex": ["1", "3", "4"],
        "comment": "One of the several ba-standards working group meeting I have attended."
        },
        {
        "id": "http://etherpad.badgealliance.org/ba-standard-2016-10-13",
        "evidenceIndex": "2",
        "criteriaIndex": ["1", "3", "4"],
        "comment": "One of the several ba-standards working group meeting I have attended."
        },
        {
        "id": "#84",
        "evidenceIndex": "3",
        "criteriaIndex": ["1", "2", "3", "4"],
        "comment": "One of my contributions to the 2.0 specs."
        }],
    },
}

User perspective: the user lists the different pieces of evidence with their individual descriptions, and what criteria they match, then uses a reference to them in the narrative (e.g. "[2]"). We should allow markup text and pictures in the narratives to make them richer.

While we have mostly considered a badge as something delivered by an institution at the end of a process, we have almost never considered the badge as the support being constructed by the earner as a means to get a recognised or earn a credential. If we use badges "as a means to earn badges" then they need to contain all the information an assessor would need to review the badge and 'endorse' it (issue/accredit).

Hence the "criteriaIndex" field to cross-reference evidence with success/performance criteria. This is particularly useful for competency based education, but not only.

[ottonomy]

Thanks for the comment, @szerge. The idea of an evidence narrative that connects the pieces of evidence is emerging as an important use case to serve. I've been thinking about a couple complications with your idea, and I identified a couple sticky points:

• Referencing links as [1], [2] requires a new parsing language that doesn't yet exist. I'd rather use Markdown Evidence item [1](http://etherpad.badgealliance.org/ba-standard-2016-11-10) shows my engagement in technical meetings...
• evidenceIndex values that don't use IRI format are odd. I can see brevity as the only advantage of using this evidenceIndex property, and machines don't care much for brevity, so why not just use the IRIs that identify each of the items.

(I'll consider the criteriaIndex in a separate comment to keep things simple).

How about the following that uses IRIs as identifiers and plain markdown as the narrative value format:

{
    "narrative": "I have contributed to the 2.0 Open Badge specification by regularly contributing to the ba-standards group call, c.f. [1](http://etherpad.badgealliance.org/ba-standard-2016-11-10) and [2](http://etherpad.badgealliance.org/ba-standard-2016-10-13), and making written suggestions, c.f. [3](https://github.com/openbadges-specification/issues/84). My goal was to make badges the building blocks of a different kind of ePortfolio, more open, modular and flexible ...",
    "evidence": [
        {
            "id": "http://etherpad.badgealliance.org/ba-standard-2016-11-10",
            "name": "Badge Alliance Standard Working Group Meeting - 10 November 2016",
            "description": "We discussed improvements to the Open Badges Assertion evidence property.",
            "comment": "One of the several ba-standards working group meetings I have attended."
        },
        {
            "id": "http://etherpad.badgealliance.org/ba-standard-2016-10-13",
            "name": "Badge Alliance Standard Working Group Meeting - 13 October 2016",
            "comment": "One of the several ba-standards working group meetings I have attended."
        },
        {
            "id": "https://github.com/openbadges-specification/issues/84",
            "name": "Referencing multiple pieces of evidence and related metadata in Assertions",
            "comment": "One of my contributions to the 2.0 specs."
        }
    ]
}

I think the only oddity there might be that the Assertion remains a document published by the issuer, and these comments come from the perspective of the recipient.

[szerge]

I agree @ottonomy that we should use IRIs as identifiers — adding an internal reference ("[XX]") was just a means to make it easy to read :-)

Re. evidenceIndex: a complete solution (esp. for competency-based badges) should have full cross referencing, i.e.:

• each piece of evidence should be connected to one or more criteria
• each criteria should be connected to one or more pieces of evidence

This could be done within the evidence field or we could have another field, crossReference that would do just that. I tend to favour the first solution to have an immediate explicit link between a piece of evidence and the associated criteria. Of course, that would require to have the criteria field structured similarly to the evidence field:

{
  "@context": "https://w3id.org/openbadges/v2",

    {
    "criteria": "In order to get that badge, you need to demonstrate
    that you are contributing to the standards working group
    [1](http://www.compedia.org/standards-design/#attending) provide
    comments [2](http://www.compedia.org/standards-design/#commenting),
    and making written suggestions, c.f.
    [3]http://www.compedia.org/standards-design/#suggesting).",
    "alignment": [
        {
            "targetCode": "Standards Design",
            "targetName": "Attending work group meetings",
            "targetUrl": "http://www.compedia.org/standards-design/#attending",
            "targetDescription": "....",
        },
        {
            "targetCode": "Standards Design",
            "targetName": "Providing comments to suggestions made by others",
            "targetUrl": "http://www.compedia.org/standards-design/#commenting",
            "targetDescription": "....",
        },
        {
            "targetCode": "Standards Design",
            "targetName": "Making suggestions to improve the group work",
            "targetUrl": "http://www.compedia.org/standards-design/#suggesting",
            "targetDescription": "....",
        }
    ]
    }
}

Re. 'oddity': even in the context of an assertion issued by the badge issuer (I would prefer using a flexible workflow where the 'accreditation' can come at the end, but that's another story), the information displayed in the badge is the information the issuer wold need in order to issue a badge as to issue a badge the following questions must be asked:

• what are the criteria?
• is the evidence produced sufficient to cover all the criteria?

then, but that's also another story:

• is it authentic (really the product of the candidate's activity)?
• is it current (wasn't it produced 3 years ago)?

For the 'other story' one solution is endorsement of evidence by identified parties (colleagues, manager, witness, client, etc.).

So, if we don't have immediately a means to manage flexible workflows, this could be implemented (hard coded) in the issuing platforms:

1. The candidate provides the information when claiming a badge (c.f. Openbadgefactory badge claim process) with all the required information: evidence, narrative, cross-reference with criteria;
2. If the issuer (assessor) is satisfied with the content of the submission form, the badge is issued.

[kayaelle]

A couple of additional properties that would be useful:

• genre or type: above schema:genre is used. This could be utilized to understand what the evidence is. Possibly make suggestions such as: text, file, url but could also be more descriptive such as "poetry" as suggested above.

• date: to show progression but also to illustrate a time period for when the evidence has been submitted

[ottonomy]

@kayaelle

genre:
One use for genre or a similar property would be to show displayer platforms several different ways they might want to embed content, rather than just a text display of "video" on the screen. But presumably they'll actually determine whether they can embed the video by inspecting the URL. (If youtube or vimeo, embed. Else, link).
I'm probably just going to include schema:genre as a property in this class as part of the proposal.

date:
Should we rename "issuedOn" to "created" so it can be a general purpose property for things that aren't "issued"?

[mgylling]

Closing this issue, as it was resolved by the 2.0 Recommendation. This issue and all other issues resolved by 2.0 can be found via the 2.0 Release Candidate milestone and PR #99.