Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor the entire test suite to make it easier to contribute implementations. #52

Merged
merged 16 commits into from
Apr 7, 2021

Conversation

msporny
Copy link
Member

@msporny msporny commented Apr 3, 2021

I have performed the following things to refactor the test suite:

  • Split did-spec into suites associated with the major sections of the test suite: identifier, core properties, consumption, production, resolution, and dereferencing.
  • Created an implementations directory that is shared among all the test suites. There can be 5 classes of implementations (so far) -- DID Methods, resolvers, dereferencers, and soon, producers and consumers.
  • Renamed and restructured the test suite output to make it more logically grouped. The general order is: major section of the spec > implementation being tested > minor section of the spec > specific variation implementation being tested > specific normative statement. It's a bit awkward, but given that test suite implementers have chosen to test different sections of the spec in different ways, it's the only logical structuring that we can do, AFAICT.
  • Unified the configuration files.

If we merge this in, it means that people submitting implementations only need to submit ONE .json file for each class of implementation they want to test. For example, if someone has a specific DID Method Document they want to submit to the test suite, all they have to do is submit one DID Document file in the implementations directory.

There are still things that need to be done:

  • Test Suite Implementers (for example for production, consumption, resolution, or URL dereferencing) may want to further re-factor their test suites given the new structure. This is optional, and we'll only know after discussing the new layout with the implementers.
  • I'm still not sure how we're generating reports from this and will have to chat with @OR13 about that.
  • I'm almost certain the "test server" part of this is broken now, but shouldn't be hard to fix. I just don't know what the architecture and goals are there, again, will need to chat with @OR13 about that.

@msporny msporny requested a review from OR13 as a code owner April 3, 2021 20:13
@OR13
Copy link
Contributor

OR13 commented Apr 7, 2021

We can scrub the test server and stick to CI for the report generation.

@msporny msporny merged commit f01a7a4 into main Apr 7, 2021
@msporny msporny deleted the msporny-refactor branch May 2, 2021 20:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants