Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: update test cases with context #344

Closed
wants to merge 1 commit into from
Closed

Conversation

jeeyyy
Copy link
Collaborator

@jeeyyy jeeyyy commented Nov 5, 2018

This PR handles adding JSON-LD(ish) context to test cases.

An example output is as below:

{
   "@context": {
    "earl": "http://www.w3.org/ns/earl#",
    "xlink": "http://www.w3.org/1999/xlink/",
    "tcdl": "http://bentoweb.org/refs/TCDL2.0/",
    "a11y-testcases": {
      "@id": "tcdl:testCaseDescription",
      "@container": "@list"
    },
    "url": "https://schema.org/URL",
    "expected": {
      "@id": "tcdl:expectedResult",
      "@type": "@vocab"
    },
    "passed": {
      "@id": "earl:passed"
    },
    "failed": {
      "@id": "earl:failed"
    },
    "inapplicable": {
      "@id": "earl:inapplicable"
    },
    "ruleId": "https://schema.org/identifier",
    "rulePage": "https://schema.org/URL"
  },
  "name": "auto-wcag test cases",
  "website": "https://auto-wcag.github.io/auto-wcag",
  "license": "https://auto-wcag.github.io/auto-wcag/license.html",
  "description": "Test Cases of auto-wcag rules",
  "a11y-testcases": [
    {
      "url": "https://auto-wcag.github.io/auto-wcag/auto-wcag-testcases/assets/SC1-1-1-image-has-name_passed_example_1.html",
      "expected": "passed",
      "ruleId": "SC1-1-1-image-has-name",
      "rulePage": "https://auto-wcag.github.io/auto-wcag/rules/SC1-1-1-image-has-name.html"
    },
    {
      "url": "https://auto-wcag.github.io/auto-wcag/auto-wcag-testcases/assets/SC1-1-1-image-has-name_passed_example_2.html",
      "expected": "failed",
      "ruleId": "SC1-1-1-image-has-name",
      "rulePage": "https://auto-wcag.github.io/auto-wcag/rules/SC1-1-1-image-has-name.html"
    },
    {
      "url": "https://auto-wcag.github.io/auto-wcag/auto-wcag-testcases/assets/SC1-1-1-image-has-name_passed_example_2.html",
      "expected": "inapplicable",
      "ruleId": "SC1-1-1-image-has-name",
      "rulePage": "https://auto-wcag.github.io/auto-wcag/rules/SC1-1-1-image-has-name.html"
    }
  ]
}

Closes issue: #333

Guidance for the PR (pull request) creator

When creating PR:

  • Make sure you requesting to pull a issue/feature/bugfix branch (right side) to the master branch (left side).

After creating PR:

  • Add yourself (and co-authors) as "Assignees" for PR
  • Add label to indicate if it's a Rule, Definition or Chore (more to the administrative side)
  • Add relevant project (e.g. "Q3 2018 Status") to PR
  • OPTIONAL: If you want anyone in particular to review your pull request, assign them as "Reviewers".
  • Close the issue that the PR resolves (and make sure the issue is referenced in the top of this comment)

How to Review And Approve

  • Go to the “Files changed” tab
  • Here you will have the option to leave comments on different lines.
  • Once the review is completed, find the “Review changes” button in the top right, select “Approve” (if you are really confident in the rule) or "Request changes" and click “Submit review”.

"@id": "earl:inapplicable"
},
"ruleId": "https://schema.org/identifier",
"rulePage": "https://schema.org/URL"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This somehow has to map to a tcdl:rule with an xlink:href attirbute. It may be that the output will have to be adjusted slightly for that to work though, e.g.:

{
  "@context": {
    "rule": {
      "@id": "tcdl:rule"
    },
    "ruleUrl": "xlink:href"
  },
  "a11y-testcases": [
    {
      ...
      "rule": {
        "ruleUrl": "https://auto-wcag.github.io/auto-wcag/rules/..."
      },
      ...
    }
  ]
}

@nitedog Might you have any idea if this can be done in a less verbose manner?

"@id": "tcdl:testCaseDescription",
"@container": "@list"
},
"url": "https://schema.org/URL",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This somehow has to map to a tcdl:file with an xlink:href attirbute. It may be that the output will have to be adjusted slightly for that to work though, e.g.:

{
  "@context": {
    "file": {
      "@id": "tcdl:file"
    },
    "fileUrl": "xlink:href"
  },
  "a11y-testcases": [
    {
      ...
      "file": {
        "fileUrl": "https://auto-wcag.github.io/auto-wcag/rules/..."
      },
      ...
    }
  ]
}

@nitedog Might you have any idea if this can be done in a less verbose manner?

@jeeyyy
Copy link
Collaborator Author

jeeyyy commented Nov 5, 2018

@nitedog - could you offer your opinion on this, before we make further changes.

@jeeyyy jeeyyy requested a review from nitedog November 5, 2018 15:53
@nitedog
Copy link
Collaborator

nitedog commented Nov 6, 2018

@kasperisager @JKODU while I appreciate reusing vocabulary, I think we are hardly using TCDL at all, so I'm not sure if adding the mappings actually helps. For example, modelling tcdl:rule and tcdl:file does not seem to make us any more aligned with TCDL, given that we do not use any of the other required parts. I'm not sure how to best do vocabulary mappings, especially since are kind of cherry-picking anyway.

@kasperisager
Copy link
Contributor

kasperisager commented Nov 12, 2018

I do see your point @nitedog. I've made an attempt at making a full JSON serialization of a test case described in TCDL and am faced with the issue that the TCDL vocabulary serializes very poorly to JSON. The primary cause seems to be the fact that TCDL defines almost every term as an element type rather than an attribute which leads to very convoluted JSON. TCDL also defines a lot of required terms, many of which I don't really see the reason for being required.

I'd love for us to rely on open data formats for talking to the outside world (testers, implementers, etc.) but if doing so introduces immense verbosity then I don't think the benefit outweighs the cost. Thoughts?

@jeeyyy
Copy link
Collaborator Author

jeeyyy commented Nov 12, 2018

I am of the opinion that this overkill for exposing the test cases.

The test cases have further been trimmed down now to not include selectors etc - https://auto-wcag.github.io/auto-wcag/auto-wcag-testcases/testcases.json

I would prefer to keep this as is, and if need arises work towards adding @context for transformation to TCDL on a later date.

Closing this PR.

@jeeyyy jeeyyy closed this Nov 12, 2018
@jeeyyy jeeyyy deleted the chore-update-testcases branch January 3, 2019 12:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants