New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Transition whatwg.org to new hosting #7

Closed
foolip opened this Issue Aug 23, 2017 · 40 comments

Comments

6 participants
@foolip

This comment has been minimized.

Show comment
Hide comment
@foolip

foolip Aug 23, 2017

Member

Separate from the hosting question, these are static sites that could be hosted inside https://github.com/whatwg/whatwg.org/:

@annevk?

Member

foolip commented Aug 23, 2017

Separate from the hosting question, these are static sites that could be hosted inside https://github.com/whatwg/whatwg.org/:

@annevk?

@annevk

This comment has been minimized.

Show comment
Hide comment
@annevk

annevk Aug 23, 2017

Member

I'd like @sideshowbarker's opinion on everything help.whatwg.org.

I'd be okay with merging whatwg/resources.whatwg.org into whatwg/whatwg.org at some point, though leaving it alone for now seems fine too? Seems somewhat more complicated if we want to keep the history intact.

For the two others I created whatwg/whatwg.org#52.

Member

annevk commented Aug 23, 2017

I'd like @sideshowbarker's opinion on everything help.whatwg.org.

I'd be okay with merging whatwg/resources.whatwg.org into whatwg/whatwg.org at some point, though leaving it alone for now seems fine too? Seems somewhat more complicated if we want to keep the history intact.

For the two others I created whatwg/whatwg.org#52.

@annevk

This comment has been minimized.

Show comment
Hide comment
@annevk

annevk Aug 23, 2017

Member

As a next step I think we should try to move everything that's not *.spec.whatwg (so we will move spec.whatwg), blog.whatwg, or wiki.whatwg. Then we could move blog.whatwg and wiki.whatwg and maybe html.whatwg (it seems like we'll continue to have a need to execute scripts so we might want to make a longer term plan first), and last all the specs (that are all intertwined I think).

Member

annevk commented Aug 23, 2017

As a next step I think we should try to move everything that's not *.spec.whatwg (so we will move spec.whatwg), blog.whatwg, or wiki.whatwg. Then we could move blog.whatwg and wiki.whatwg and maybe html.whatwg (it seems like we'll continue to have a need to execute scripts so we might want to make a longer term plan first), and last all the specs (that are all intertwined I think).

@foolip

This comment has been minimized.

Show comment
Hide comment
@foolip

foolip Aug 23, 2017

Member

Yeah, let's leave resources.whatwg.org alone, because it's used by HTML it might get more traffic, so might not want it deployed in the same place. Will you put images.whatwg.org and n.whatwg.org into the repo though? We can set up the deploy for that before DNS is changed.

Member

foolip commented Aug 23, 2017

Yeah, let's leave resources.whatwg.org alone, because it's used by HTML it might get more traffic, so might not want it deployed in the same place. Will you put images.whatwg.org and n.whatwg.org into the repo though? We can set up the deploy for that before DNS is changed.

@foolip

This comment has been minimized.

Show comment
Hide comment
@foolip
Member

foolip commented Aug 23, 2017

Oops, that's whatwg/whatwg.org#52

@sideshowbarker

This comment has been minimized.

Show comment
Hide comment
@sideshowbarker

sideshowbarker Aug 23, 2017

Member

help.whatwg.org can just be redirected

Member

sideshowbarker commented Aug 23, 2017

help.whatwg.org can just be redirected

@foolip

This comment has been minimized.

Show comment
Hide comment
@foolip

foolip Aug 24, 2017

Member

resources.whatwg.org is a tricky one, because it's used by all specs. If it's not on the same IP and using the same cert as the specs, then load time will be less awesome than it could be. Maybe we should try hosting everything static on a single machine, and just make it beefier?

Member

foolip commented Aug 24, 2017

resources.whatwg.org is a tricky one, because it's used by all specs. If it's not on the same IP and using the same cert as the specs, then load time will be less awesome than it could be. Maybe we should try hosting everything static on a single machine, and just make it beefier?

@annevk

This comment has been minimized.

Show comment
Hide comment
@annevk

annevk Aug 24, 2017

Member

I think we should host all static things on the same server instance. But note that any performance benefits won't come until Let's Encrypt actually offers multi-domain wildcard certificates.

Member

annevk commented Aug 24, 2017

I think we should host all static things on the same server instance. But note that any performance benefits won't come until Let's Encrypt actually offers multi-domain wildcard certificates.

@foolip

This comment has been minimized.

Show comment
Hide comment
@foolip

foolip Aug 24, 2017

Member

We could also request a single cert for all the domains on a single machine.

Member

foolip commented Aug 24, 2017

We could also request a single cert for all the domains on a single machine.

@foolip

This comment has been minimized.

Show comment
Hide comment
@foolip

foolip Aug 24, 2017

Member

The downside is that if the DNS of any one of those domains changes, I think certbot renew would start failing.

Member

foolip commented Aug 24, 2017

The downside is that if the DNS of any one of those domains changes, I think certbot renew would start failing.

@foolip

This comment has been minimized.

Show comment
Hide comment
@foolip

foolip Aug 24, 2017

Member

The current setup for spec.whatwg.org (nested inside whatwg.org source-wise) leads to fun:
https://whatwg.org/specs/web-forms/current-work/
https://spec.whatwg.org/web-forms/current-work/

These both work, but the latter looks like an accident. I suggest that we just let those URLs go 404, and preserve only the original URLs.

Member

foolip commented Aug 24, 2017

The current setup for spec.whatwg.org (nested inside whatwg.org source-wise) leads to fun:
https://whatwg.org/specs/web-forms/current-work/
https://spec.whatwg.org/web-forms/current-work/

These both work, but the latter looks like an accident. I suggest that we just let those URLs go 404, and preserve only the original URLs.

@annevk

This comment has been minimized.

Show comment
Hide comment
@annevk

annevk Aug 24, 2017

Member

Yes, we should do that.

Member

annevk commented Aug 24, 2017

Yes, we should do that.

@foolip

This comment has been minimized.

Show comment
Hide comment
@foolip

foolip Aug 25, 2017

Member

I'm extracting out spec.whatwg.org now.

Member

foolip commented Aug 25, 2017

I'm extracting out spec.whatwg.org now.

@foolip

This comment has been minimized.

Show comment
Hide comment
@foolip

foolip Aug 25, 2017

Member

Handing html-differences in whatwg/html-differences#5

Member

foolip commented Aug 25, 2017

Handing html-differences in whatwg/html-differences#5

@foolip

This comment has been minimized.

Show comment
Hide comment
@foolip

foolip Aug 30, 2017

Member

I've added lists.whatwg.org to the list, it's currently broken by HSTS, and it would be nice to restore it somehow.

Member

foolip commented Aug 30, 2017

I've added lists.whatwg.org to the list, it's currently broken by HSTS, and it would be nice to restore it somehow.

@annevk

This comment has been minimized.

Show comment
Hide comment
@annevk

annevk Aug 30, 2017

Member

@foolip I suspect we can't get ownership of that until we have DNS control since it's highly likely a name that's tightly coupled to DreamHost's managed mailing list service. (When we take that over we should make sure that whatwg@lists.whatwg.org still works. Same for whatwg@whatwg.org.)

Member

annevk commented Aug 30, 2017

@foolip I suspect we can't get ownership of that until we have DNS control since it's highly likely a name that's tightly coupled to DreamHost's managed mailing list service. (When we take that over we should make sure that whatwg@lists.whatwg.org still works. Same for whatwg@whatwg.org.)

@foolip

This comment has been minimized.

Show comment
Hide comment
@foolip

foolip Aug 30, 2017

Member

Right, so the challenge is how to get the archives from DreamHost and, presumably, to keep updating them if we're not shutting down all of the lists.

Member

foolip commented Aug 30, 2017

Right, so the challenge is how to get the archives from DreamHost and, presumably, to keep updating them if we're not shutting down all of the lists.

@annevk

This comment has been minimized.

Show comment
Hide comment
@annevk

annevk Aug 30, 2017

Member

Well, the main list we'd end up keeping is also archived at https://lists.w3.org/Archives/Public/public-whatwg-archive/. If we can it might be better to just find a mapping from the old URL to those URLs and redirect everything there. The alternative would be to install the same mailing list software (unless we want to deal with local redirects).

Member

annevk commented Aug 30, 2017

Well, the main list we'd end up keeping is also archived at https://lists.w3.org/Archives/Public/public-whatwg-archive/. If we can it might be better to just find a mapping from the old URL to those URLs and redirect everything there. The alternative would be to install the same mailing list software (unless we want to deal with local redirects).

@foolip

This comment has been minimized.

Show comment
Hide comment
@foolip

foolip Aug 30, 2017

Member

That would work, if https://lists.w3.org/Archives/Public/public-whatwg-archive/ contains everything that our own archives do. Still probably a lot of work.

Member

foolip commented Aug 30, 2017

That would work, if https://lists.w3.org/Archives/Public/public-whatwg-archive/ contains everything that our own archives do. Still probably a lot of work.

@annevk

This comment has been minimized.

Show comment
Hide comment
@annevk

annevk Aug 30, 2017

Member

It should be complete, but yes.

Member

annevk commented Aug 30, 2017

It should be complete, but yes.

@foolip

This comment has been minimized.

Show comment
Hide comment
@foolip

foolip Sep 4, 2017

Member

@domenic, is any action needed for build.whatwg.org? At the very least we shouldn't mess up its DNS in eventual DNS handover.

Member

foolip commented Sep 4, 2017

@domenic, is any action needed for build.whatwg.org? At the very least we shouldn't mess up its DNS in eventual DNS handover.

@domenic

This comment has been minimized.

Show comment
Hide comment
@domenic

domenic Sep 4, 2017

Member

It might be a good idea for me to move that all from my personal AWS to one of the droplets. It'd also be good to switch to LetsEncrypt.

If you're up for it, I'd appreciate the help doing so; the readme at https://github.com/domenic/wattsi-server should have most of the info. Then you could switch the DNS to point to the droplet and I could shut down my server completely.

Eventually we probably want to move that to a whatwg/build.whatwg.org repository.

Member

domenic commented Sep 4, 2017

It might be a good idea for me to move that all from my personal AWS to one of the droplets. It'd also be good to switch to LetsEncrypt.

If you're up for it, I'd appreciate the help doing so; the readme at https://github.com/domenic/wattsi-server should have most of the info. Then you could switch the DNS to point to the droplet and I could shut down my server completely.

Eventually we probably want to move that to a whatwg/build.whatwg.org repository.

@foolip

This comment has been minimized.

Show comment
Hide comment
@foolip

foolip Sep 5, 2017

Member

I've added it to the list so that it's not forgotten.

Member

foolip commented Sep 5, 2017

I've added it to the list so that it's not forgotten.

@foolip

This comment has been minimized.

Show comment
Hide comment
@foolip

foolip Nov 7, 2017

Member

What's left blocking HTML is now print.pdf. Should try to make https://github.com/izh1979/makepdf part of the Travis build. Thanks @izh1979 for putting it together!

Member

foolip commented Nov 7, 2017

What's left blocking HTML is now print.pdf. Should try to make https://github.com/izh1979/makepdf part of the Travis build. Thanks @izh1979 for putting it together!

@foolip

This comment has been minimized.

Show comment
Hide comment
@foolip

foolip Nov 7, 2017

Member

So I'm thinking maybe we could just build/install https://github.com/izh1979/makepdf on Travis for each commit?

Member

foolip commented Nov 7, 2017

So I'm thinking maybe we could just build/install https://github.com/izh1979/makepdf on Travis for each commit?

@foolip

This comment has been minimized.

Show comment
Hide comment
@foolip

foolip Nov 7, 2017

Member

The other alternative would be to make it part of build.whatwg.org, but then we'd have to worry about malicious input, and not just processing what gets merged into the spec.

Member

foolip commented Nov 7, 2017

The other alternative would be to make it part of build.whatwg.org, but then we'd have to worry about malicious input, and not just processing what gets merged into the spec.

@annevk

This comment has been minimized.

Show comment
Hide comment
@annevk

annevk Nov 7, 2017

Member

The more we can outsource to Travis the better I think.

Member

annevk commented Nov 7, 2017

The more we can outsource to Travis the better I think.

@domenic

This comment has been minimized.

Show comment
Hide comment
@domenic

domenic Nov 7, 2017

Member

Yes, putting it in the Docker stuff for CI build is my plan. It sounds like I should work on that sooner rather than later.

Member

domenic commented Nov 7, 2017

Yes, putting it in the Docker stuff for CI build is my plan. It sounds like I should work on that sooner rather than later.

@foolip

This comment has been minimized.

Show comment
Hide comment
@foolip

foolip Nov 9, 2017

Member

Yeah, otherwise we'll have to either run PHP on the machine serving HTML, or on some other machine that accepts the PDF and deploys it.

Getting the pdfbuild repo working on Travis isn't trivial, but is probably possible in the end.

Member

foolip commented Nov 9, 2017

Yeah, otherwise we'll have to either run PHP on the machine serving HTML, or on some other machine that accepts the PDF and deploys it.

Getting the pdfbuild repo working on Travis isn't trivial, but is probably possible in the end.

@izh1979

This comment has been minimized.

Show comment
Hide comment
@izh1979

izh1979 Nov 9, 2017

Or you can rewrite the script in some other language. ;-) It's not big, nor complex.
And it's easier when you have working variant. ;-)

izh1979 commented Nov 9, 2017

Or you can rewrite the script in some other language. ;-) It's not big, nor complex.
And it's easier when you have working variant. ;-)

@snuggs

This comment has been minimized.

Show comment
Hide comment
@snuggs

snuggs Nov 10, 2017

Member

@izh1979 @foolip where would said script be located? Or is this related to whatwg/html-build#120 (comment) & whatwg/html-build#122

Lastly, what is http://sgr-a.ru/h/whatwgpdf.php ?

P.S. @annevk 👍 for PDF'ing the standards.

Thanks in advance!

Member

snuggs commented Nov 10, 2017

@izh1979 @foolip where would said script be located? Or is this related to whatwg/html-build#120 (comment) & whatwg/html-build#122

Lastly, what is http://sgr-a.ru/h/whatwgpdf.php ?

P.S. @annevk 👍 for PDF'ing the standards.

Thanks in advance!

@annevk

This comment has been minimized.

Show comment
Hide comment
@annevk

annevk Nov 10, 2017

Member

@snuggs the script used for the HTML Standard is hosted at https://github.com/izh1979/makepdf and that .php page runs it.

Member

annevk commented Nov 10, 2017

@snuggs the script used for the HTML Standard is hosted at https://github.com/izh1979/makepdf and that .php page runs it.

@foolip

This comment has been minimized.

Show comment
Hide comment
@foolip

foolip Nov 11, 2017

Member

Well, this is amusing, while trying to figure out how to do this, I came across https://www.princexml.com/forum/topic/2666/using-better-compression-and-pdf-size-optimization which is @izh1979 asking about compressing PDFs :)

Member

foolip commented Nov 11, 2017

Well, this is amusing, while trying to figure out how to do this, I came across https://www.princexml.com/forum/topic/2666/using-better-compression-and-pdf-size-optimization which is @izh1979 asking about compressing PDFs :)

@izh1979

This comment has been minimized.

Show comment
Hide comment
@izh1979

izh1979 Nov 13, 2017

Yeah, I've generating PDFs for WHATWG HTML spec for about of 3 years. ;-)

izh1979 commented Nov 13, 2017

Yeah, I've generating PDFs for WHATWG HTML spec for about of 3 years. ;-)

foolip added a commit to whatwg/html-build that referenced this issue Dec 1, 2017

foolip added a commit to whatwg/whatwg.org that referenced this issue Dec 5, 2017

foolip added a commit to whatwg/whatwg.org that referenced this issue Dec 5, 2017

@foolip

This comment has been minimized.

Show comment
Hide comment
@foolip

foolip Dec 11, 2017

Member

All that remains here now is lists.whatwg.org, which is tied together with the DNS. Somebody needs to learn how email actually works to do this job, maybe we should liaison with the IETF :)

Member

foolip commented Dec 11, 2017

All that remains here now is lists.whatwg.org, which is tied together with the DNS. Somebody needs to learn how email actually works to do this job, maybe we should liaison with the IETF :)

foolip added a commit to whatwg/html that referenced this issue Dec 15, 2017

foolip added a commit to whatwg/whatwg.org that referenced this issue Dec 15, 2017

foolip added a commit to whatwg/whatwg.org that referenced this issue Dec 15, 2017

annevk added a commit to whatwg/html that referenced this issue Dec 16, 2017

@foolip

This comment has been minimized.

Show comment
Hide comment
@foolip

foolip Dec 17, 2017

Member

An update on lists.whatwg.org. https://packages.debian.org/sid/mailman3-suite looked very promising, but is only in Debian unstable, and installation didn't actually work out of the box on a fresh Debian unstable, neither an (upgraded) DigitalOcean droplet or in a debian:unstable Docker image. (I think it failed to create/start a postgres db, and is probably easy-ish to fix.)

Because Debian unstable also doesn't get security updates in the same way, it's probably best to follow http://docs.mailman3.org/en/latest/prodsetup.html or similar, and 213da82 is what I tried so far. With some more tinkering it's probably possible to make that work.

I haven't tried https://github.com/maxking/docker-mailman yet because it seems somewhat implausible to me that the total complexity will be lower, one still has to have a database and web server separately, and we don't want archives, so it'd pretty much just give us mailman3-core and the django stuff.

On setting up an email server (not just mailman), something very discouraging at https://www.digitalocean.com/community/tutorials/how-to-set-up-a-postfix-e-mail-server-with-dovecot:

Postfix has several hundred configuration parameters. If you want to administer a mail server that reliably delivers business requirements to a sizable organization, you should make yourself intimate with all of them (man 5 postconf). This tutorial will not be enough, on its own, to make you a competent professional email admin.

Member

foolip commented Dec 17, 2017

An update on lists.whatwg.org. https://packages.debian.org/sid/mailman3-suite looked very promising, but is only in Debian unstable, and installation didn't actually work out of the box on a fresh Debian unstable, neither an (upgraded) DigitalOcean droplet or in a debian:unstable Docker image. (I think it failed to create/start a postgres db, and is probably easy-ish to fix.)

Because Debian unstable also doesn't get security updates in the same way, it's probably best to follow http://docs.mailman3.org/en/latest/prodsetup.html or similar, and 213da82 is what I tried so far. With some more tinkering it's probably possible to make that work.

I haven't tried https://github.com/maxking/docker-mailman yet because it seems somewhat implausible to me that the total complexity will be lower, one still has to have a database and web server separately, and we don't want archives, so it'd pretty much just give us mailman3-core and the django stuff.

On setting up an email server (not just mailman), something very discouraging at https://www.digitalocean.com/community/tutorials/how-to-set-up-a-postfix-e-mail-server-with-dovecot:

Postfix has several hundred configuration parameters. If you want to administer a mail server that reliably delivers business requirements to a sizable organization, you should make yourself intimate with all of them (man 5 postconf). This tutorial will not be enough, on its own, to make you a competent professional email admin.

@domenic

This comment has been minimized.

Show comment
Hide comment
@domenic

domenic Dec 17, 2017

Member

Oh, wow, thanks for looking into this further. Can you explain a bit more about the docker-mailman not working? It seems to have the database inside the container.

Member

domenic commented Dec 17, 2017

Oh, wow, thanks for looking into this further. Can you explain a bit more about the docker-mailman not working? It seems to have the database inside the container.

@foolip

This comment has been minimized.

Show comment
Hide comment
@foolip

foolip Dec 17, 2017

Member

I haven't tried setting it up the Docker image. From my reading of https://hub.docker.com/r/alexbarcelo/mailman3/ last time, given the POSTGRES_HOST configuration I surmised that the database would be external, but maybe it'll use an internal one if not.

Member

foolip commented Dec 17, 2017

I haven't tried setting it up the Docker image. From my reading of https://hub.docker.com/r/alexbarcelo/mailman3/ last time, given the POSTGRES_HOST configuration I surmised that the database would be external, but maybe it'll use an internal one if not.

@foolip

This comment has been minimized.

Show comment
Hide comment
@foolip

foolip Jan 8, 2018

Member

Filed #75 for what remains.

Member

foolip commented Jan 8, 2018

Filed #75 for what remains.

@foolip foolip closed this Jan 8, 2018

@domenic

This comment has been minimized.

Show comment
Hide comment
@domenic

domenic Jan 8, 2018

Member

Also filed #77

Member

domenic commented Jan 8, 2018

Also filed #77

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment