Update standards landing pages to conform to Standards Documentation Specification

[baskaufs]

Please note: creating this pull request is NOT a request to merge (yet). When all of the myriad related issues have been checked and relevant people have signed off, it can be merged.

Interested parties: @timrobertson100 @stanblum @peterdesmet @nickynicolson @jmacklin and probably others.

This pull request is intended to deal with the following problems:

1. Standards have been conflated with standards documents. In some cases a standard has only a single document, but in many cases, standards have multiple documents, often with different authors. The SDS requires in Section 3.1.7 that the standards landing page list and link all parts of the standard.
2. Some of the older TDWG standards did not have any landing page or had pages with a lot of missing information (such as where to find a copy of the standard).
3. The metadata on the standards landing page needs to agree with the machine-readable metadata that is available for standards and documents. The solution to this is to build the landing pages from the same CSV tables in the rs.tdwg.org repo that is the source of the machine-readable metadata.
4. The headers on the existing pages don't entirely conform to Section 3.1

I have used a Python script that draws from the rs.tdwg.org repo and from a page info CSV that contains information from existing landing pages (images, headers, extra text, etc.). The script generates the draft pages in an output directory that mirrors the structure of the standards pages directory in the website repo. After generating the drafts, they can be copied over to the corresponding website directory to replace the existing page.

This pull request addresses some of the checkboxes in tdwg/infrastructure#93 (comment), and should close #129, #125, #126, #124, #123, #121, and #46.

There are also some issues in the rs.tdwg.org tracker that will be addressed here. They are grouped in https://github.com/tdwg/rs.tdwg.org/milestone/1. Since these issues are generally complex, I'll address them in separate comments.

There are a number of the previously hidden stub pages that aren't fancied up with background images. Personally I don't care about that, but in the interest of maintaining a consistent look for the site, we probably should add some. Who picks those out? If we want to add some, it would be better/easier to just put them into the page info CSV rather than hand editing. It takes about 0.1 second to rerun the script that builds the pages.

I put a lot of effort into trying to figure out what documents actually need to be part of the older standards and also into figuring out who the authors were. I have undoubtedly made mistakes that will need to be fixed. Again, it is better to fix the info in the source tables in rs.tdwg.org than to do manual page editing -- that is important because manually editing the page won't have any effect on the machine-readable data. If you find errors, let me know and I can fix the sources and run them through the pipeline again.

[baskaufs]

@DavidFichtmueller @wouteraddink @WalterBerendsohn In #368 I am working on updating all of the standards landing pages to bring them into conformance with the Standards Documentation Specification. The pages are being generated from tables and not build by hand, but that required sorting out what particular documents were officially considered part of the standard (mostly ones that would be considered to have normative content).

I struggled with ABCD because there were a lot of documents around at the time I sorted through them a couple years ago. The draft page generated from the data I have at the moment is here. It doesn't include anything about 3.0. I'm not sure exactly how you are considering the versions to work (i.e. does 3.0 replace 2.06 or should they just be considered to have separate documents that are all included in the current standard?). So if you can give me a list of the docs (and their URLs) that you consider to be part of the standard, I'll add them.

Also, the draft page doesn't have any information about how ABCD operates or a link to the ABCD website https://abcd.tdwg.org/ in the text area. I should add that if you can tell me how you would like for it to look.

The draft page isn't rendered using the page template, but its style should look like the existing page https://www.tdwg.org/standards/abcd/ when it gets merged.

(Refer: tdwg/rs.tdwg.org#8)

[baskaufs]

@gmhagedorn @BryanHeidorn In #368 I'm working on updating all of the TDWG standards landing pages to bring them into conformance with the Standards Documentation Specification. One issue I faced was figuring out what documents should actually be considered part of the older standards. Can you take a look at this page and the two documents I've listed as part of SDD. Are they correct? Are there other documents I'm missing?

(Refer: tdwg/rs.tdwg.org#8)

[baskaufs]

@regiov @mdoering @tucotuco (and others) In #368 I'm working on updating all of the TDWG standards landing pages to bring them into conformance with the Standards Documentation Specification. The draft updated page mostly contains the existing information that's on the existing TAPIR landing page. What I have added is the "Parts of the standard" section, in which I attempted to enumerate what documents were actually part of the standard itself. Any document that would be considered to contain normative content should be there -- purely informative documents would not necessarily be considered to be "within" the standard.

Can you please review this list for omissions or docs I put there that should not be included? Also, it took a bit of detective work to figure out all the authors, affiliations, etc. so any QC on the existing metadata would be appreciated.

(Refer: tdwg/rs.tdwg.org#8)

[baskaufs]

@stanblum @ArthurChapman @chicoreus @deepreef @gkampmeier @ghwhitbread @Tasilee @WUlate In #368 I'm working on updating all of the TDWG standards landing pages to bring them into conformance with the Standards Documentation Specification. One of the most embarrassing aspects of the whole effort is that as far as I can tell TDWG has lost one of its standards: XDF. See tdwg/rs.tdwg.org#4 for the details.

Do any of you have any idea where a copy of the actual standard (not the paper published about the standard) is, or know of other long-term TDWG participants who might know? I'm sure somebody must have it on a hard drive somewhere, but a few years ago I wasted a bunch of time looking for it on the Internet unsuccessfully. It also might be on the Wayback Machine (Internet Archive) if one knew a URL to check.

The draft updated standard landing page doesn't have any link to a normative document (the URL redirects to the published paper). That may be the best we can do, but if it's really the case that nobody knows where this standard is, then it would definitely be a candidate to move to the "Retired Standard" category!

(Refer: tdwg/rs.tdwg.org#8)

[baskaufs]

@debpaul In #368 I'm working on updating all of the TDWG standards landing pages to bring them into conformance with the Standards Documentation Specification. A key bit is sorting out how to acknowledge contributors to standards and to documents within the standards. I wrote a detailed analysis of this problem at tdwg/rs.tdwg.org#24 several years ago. Since then, I have sorted out the acknowledgement of contributors in a manner that satisfies me (see the draft DwC landing page for example with data drawn from this table of standards contributors). Mostly this means listing the contributors and their roles for each document in a standard rather than trying to make any attributions at the standard level.

However, it is still unclear to me what's appropriate for the recommended citations. Generally, I have the citations for documents give the Task/Interest Group as the author. I guess that's OK and I think that's what the SDS says to do, but if people really follow that citation form, then the contributions of the actual authors don't get acknowledged. For some of the older standards, it's a lot murkier who the actual authors are.

Someone told me that the Executive has some group looking into the issue of how we do citations. If you are able to direct them to tdwg/rs.tdwg.org#24 in particular and the draft standards landing pages in this directory, I'd appreciate any feedback. I hope that this cleanup/pull request gets finished long before the citation issue gets worked out, but in the long run I'd like to bring all of the citations in the standards and documents metadata be updated to whatever is considered TDWG best practice.

[baskaufs]

Alright, I've reviewed all of the issues I could find related to this pull request and closed a number that are no longer relevant. I think that tdwg/infrastructure#93 encapsulates or links to all of the relevant issues that remain.

We can give some time for responses from people I've pinged in the comments above and assess how close this is to being ready to merge.

[tucotuco]

What you have done for TAPIR looks good to me. The approval of @regiov or @mdoering would be definitive.

[baskaufs]

Need confirmation from @jmacklin and @nielsklazenga that this is true prior to merging

[baskaufs]

@timrobertson100 About two weeks have passed since I created this pull request and with one exception (thanks @tucotuco) I have gotten no responses or feedback. I take that as a sign that no one is disturbed enough by what I've done here to block doing the merge. The questions that I raised above still remain, but there is no reason why they can't be dealt with on a case-by-case basis after the new standards pages go live. Similarly, if anyone is unhappy with page styling, lack of banner images, etc. they can fix that later. Add the icing after the cake is baked as it were.

As we had discussed in the past, this is one of the last major places where the Standards Documentation Specification has not yet been implemented. The changes I have made here will fix that and close a large number of issues that have been hanging open for years. I do not want to see this pull request languish much longer since that would just increase the probability of merge conflicts that would have to be resolved.

So if as Infrastructure Chair you concur, please review and perform the merge. (I can't request a review since I don't have write access to this repo.) See these diffs for the major changes that would occur.

Thanks,
Steve

[stanblum]

This is a fantastic step forward!