Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add a ratingExplanation property on Rating (and highlight factcheck / ClaimReview usecase) #2300

Open
danbri opened this issue Jul 10, 2019 · 4 comments

Comments

@danbri
Copy link
Contributor

commented Jul 10, 2019

It is sometimes useful for publishers of ratings to provide a brief supporting explanation, giving context and background to the rating decision. This arises in particular for fact checking, which uses the ClaimReview type (a subtype of Review) and an associated Rating which is often but not always numeric.

  • a new Property named "ratingExplanation", expected on Rating, with Text value
  • definition: "A short explanation (e.g. one to two sentences) providing background context, facts and information that led to the conclusion expressed in the rating."

This idea was discussed in the fact checking community during Check & Tech, and Global Fact events in 2019, and is proposed here as something to be experimentally implemented. So it would go into Pending area initially.

@danbri danbri self-assigned this Jul 10, 2019

danbri added a commit that referenced this issue Jul 10, 2019

danbri added a commit that referenced this issue Jul 10, 2019

@danbri

This comment has been minimized.

Copy link
Contributor Author

commented Jul 10, 2019

draft staged on webschemas for review: https://webschemas.org/ratingExplanation

"A short explanation (e.g. one to two sentences) providing background context and other information that led to the conclusion expressed in the rating. This is particularly applicable to ratings associated with "fact check" markup using ClaimReview."

An example scenario from Duke Reporter's Lab, using Politifact:

this example from Duke Reporter’s Lab Squash Fact Checker (using Politifact Fact Check):

  • claim text: “In the last two years, ICE officers made 266,000 arrests of aliens with…”
  • rating text: “Inflates the numbers.”
  • so the “context/explanation” (i.e. ratingExplanation) for the rating: “The numbers need context. Yes, during the last two years ICE officials arrested 266,000 people. But ICE notes that each arrest ‘may represent multiple criminal charges and convictions,’thus duplicating criminal activity and inflating the numbers”

The suggestion is that this new field will give a place for such information to be made machine-readably accessible.

@danbri

This comment has been minimized.

Copy link
Contributor Author

commented Aug 1, 2019

This is live in today's Schema.org 3.9 release. Let's keep this ticket open for discussion / feedback.

@DukeReportersLab

This comment has been minimized.

Copy link

commented Aug 15, 2019

We proposed this idea at the Duke Tech & Check conference, so it's great to see it now in open for feedback.

We work with the fact-checking organizations that will be using this field, so we are aware of the demands on the people who will fill out this field. We also are developing apps that use ClaimReview, so we're aware of its value and the importance that it be concise and clear.

We will be doing some initial testing in the Reporters' Lab using previously published fact-checks on the best format for the content of this field. Should the ratingExplanation be limited to a certain number of words? Should it be one sentence? Two sentences? We'll post the results of our tests here.

@cguess

This comment has been minimized.

Copy link

commented Aug 16, 2019

@danbri The only suggestion I have at this point is if we can indicate whether or not there is a technical limit to the number of characters in the documentation. The current description says:

A short explanation (e.g. one to two sentences)

However, if there's a technical limit it would be good to document that, and to do the same if there is not.

If there is no technical limit (though I imagine that there is, simply due to the database type used internally), we should probably discuss whether specifications should indicate a maximum limit. It will help give marching orders to other implementations of this proposal so they follow suit with Google's version or vice versa.

My vote is on this limit being slightly outrageously large, like 5000 characters or something. This way users can decide on how they want to have it, while also indicating to other organizations what they should at least support technically. With the current language I could easily see a project manager who speaks German saying "oh, one or two sentences, that's like 20 words," capping it arbitrarily, and breaking when they try to scrape an organization that is publishing in English or French.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
3 participants
You can’t perform that action at this time.