No good way to automate publication from GitHub without using the gh-pages branch

[tidoust]

I'm filing this issue against Echidna although the solution might be a change to one of its components (Specberus or the CSS validator, typically).

Context

We switched to using Echidna for the Presentation API in the Second Screen Working Group. We are not using ReSpec and are instead generating the spec ourselves.

To avoid confusing people with "Working Drafts" that would appear on https://w3c.github.io/presentation-api/, we decided to use a dedicated TR branch for Working Drafts that are to be published in TR space and thus to restrict the gh-pages branch to Editor's Drafts. That does not work well with Echidna, though.

Problem

As GitHub only "publishes" the gh-pages branch, we can but pass the raw content of the specification to Echidna, typically: https://raw.githubusercontent.com/w3c/presentation-api/TR/releases/WD.html

That raw content is served with a text/plain MIME type. This bugs the CSS validator used by Specberus when it checks the document as it would like to see text/html, see validation.css error in:
https://labs.w3.org/pubrules/?url=https%3A%2F%2Fraw.githubusercontent.com%2Fw3c%2Fpresentation-api%2FTR%2Freleases%2FWD.html&profile=WD-Echidna&validation=simple-validation&noRecTrack=false&informativeOnly=false&echidnaReady=true&patentPolicy=pp2004&processDocument=2014

To work around the problem, we had to go through a third-party service such as RawGit that serves the content with the right MIME type. This works fine but most (if not all) of these services use caching (for instance, RawGit says "New changes you push to GitHub will be reflected within minutes"), so I expect that we might soon run into weird errors where Echidna will not process the latest version of the document to publish.

Being able to use the raw content served from raw.githubusercontent.com directly would solve the problem.

Possible solutions

Is there a way for Specberus to tell the CSS validator that the document should be regarded as text/html (that does not seem to be possible looking at the CSS validator documentation though)?

If not, could we setup a service à la RawGit on some W3C server, perhaps restricted to W3C repositories on GitHub to avoid abuse, that would not use caching?

Or, could Echidna install the document(s) to publish locally and pass that document before running the checks?

Any other suggestion?

[mfoltzgoogle]

We could have the CSS validator respect a content type passed via a <meta> tag in the document, e.g.

<meta http-equiv="content-type" content="text/html; charset=UTF-8">

[tripu]

Maybe the easiest solution would be to pass along the text (not the URI) of the document to the CSS validator from Echidna (your last suggestion, @tidoust). Using the parameter text. After all, Echidna has the CSS in its side already.

I'll try to fix this soon.

[tripu]

@tidoust,

We switched to using Echidna for the Presentation API in the Second Screen Working Group. We are not using ReSpec and are instead generating the spec ourselves.

Now that we have support for TAR in Echidna, and given that you're already generating the spec at your end, would it be feasible for your team to simply put that spec inside a TAR file and give that URL to Echidna? There would be no problem with MIME types etc then, and Echidna doesn't care where that TAR lives (GH pages, or some other server)…

[tidoust]

@tripu,

Thanks! Support for TAR would indeed have solved the problem for us, but I realize I should have updated this issue to note that the issue no longer affects the Second Screen WG: the group eventually switched over to ReSpec, which is well integrated with Echidna.

I'm closing this issue accordingly (not sure if you follow a more formal process for issues, just re-open the issue if you do ;))

[tripu]

Ah, OK then, @tidoust!

(I've already conducted the formal process of dancing three times counter-clockwise around this issue, so all is good ;)