Deduplicating deploy scripts #6

Closed
domenic opened this Issue Jan 14, 2017 · 14 comments

Projects

None yet

3 participants

@domenic
Member
domenic commented Jan 14, 2017 edited

Related to #202, it's a bit annoying for every repo's commit history to contain a bunch of meta commits every time we want to update the deploy process. And, code duplication in general is bad.

Ideally we'd use the .travis.yml to set config variables:

  • DEPLOY_USER
  • TITLE
  • SHORTNAME (=> current LS_URL, COMMIT_URL_BASE, BRANCH_URL_BASE, SERVER, WEB_ROOT)
  • INPUT_FILE (or, just standardize on a single filename, e.g. index.bs or specshortname.bs... would lose a lot of Git history though, so probably not worth it)
  • FILES_GLOB (to allow copying extra files, like images or encoding stuff)
  • EMU_ALGIFY (boolean used by streams and console to run through emu-algify)

I guess SHORTNAME could be inferred from some git commands to get the current repository and INPUT_FILE could be inferred to be the only .bs file in the repo.

We could then I guess have a simple one- or two-liner in Travis that curls down the deploy script from whatwg/meta and runs it.

I might try prototyping this early next week; feel free to start without me though (a good first pass would probably leave out FILES_GLOB and EMU_ALGIFY)

@domenic
Member
domenic commented Jan 14, 2017

We may also be able to roll the validator stuff into the deploy script

@annevk annevk referenced this issue Jan 14, 2017
Closed

Using SSH securely #5

@annevk
Member
annevk commented Jan 15, 2017

I guess the only thing that would go away with this is local deployment. I guess that's not a big deal?

@annevk
Member
annevk commented Jan 15, 2017

I tried something at whatwg/url#205 but I'm again hitting the limitations of Travis documentation.

@annevk
Member
annevk commented Jan 15, 2017

Got it to work after taking a break. Things I'd like someone else to take on:

  • Making INPUT_FILE default to "$SHORTNAME.bs" if not given.
  • Performing HTML checking in deploy.sh
  • Deciding where to host the shared script and put it there. resources.whatwg.org?
@domenic
Member
domenic commented Jan 15, 2017 edited

Here would be my to-do for early next week, building on your work:

  • Making INPUT_FILE default to "find the single .bs file in this directory"
  • Making SHORTNAME default to the Git repo name
  • Performing HTML checking, including downloading the checker, in deploy.sh
  • Hosting the shared script on either resources.whatwg.org or whatwg/meta
  • Hosting the known_hosts file in the same place and curling it down, instead of writing it out inside deploy.sh
  • Making copying of .htaccess optional (i.e. the build should not fail if the repo does not have .htaccess checked in)
  • Figuring out how to get some kind of local deploy working again (e.g. the Makefile copies down the deploy script)

I think whatwg/meta is better than resources.whatwg.org probably... not sure though, both are kind of overextending their boundaries. Maybe we should create a whatwg/build.

@annevk
Member
annevk commented Jan 16, 2017

What is the advantage of loading known_hosts each time? As long as we have a single server, it will only need a single entry.

@domenic
Member
domenic commented Jan 16, 2017

What is the advantage of loading known_hosts each time?

I don't understand this question. Why do you think my version is different from your version? Your version codes it into the build-script; mine allows it to be in a separate file. But they both load known_hosts the same number of times. It just seems cleaner to not inline files into other files with echo statements.

As long as we have a single server, it will only need a single entry.

Right, "as long as we have a single server". But we don't: we have two right now (one for streams/console/a couple others, and one for everything else). And in general IMO it will be simpler to list all the hostnames in the file, with their corresponding keys, as that will make it easier to shift between servers in the future.

@annevk
Member
annevk commented Jan 16, 2017

Well, if you curl known_hosts from the build script, that's an additional fetch for Travis. If you inline it you only need to fetch the build script. And I'm pretty sure we have one server, just different users on that server.

@domenic
Member
domenic commented Jan 16, 2017

Hmm, I see. I'm not really concerned about multiple fetches, but maybe given that we have a single server it's simpler to just inline it... it's more of a pain to dig through the file and update, I guess, but oh well.

@annevk
Member
annevk commented Jan 16, 2017

If you really want a distinct resource, we could also use zip/unzip to avoid two fetches (until HTTP/2 is everywhere anyway).

@domenic
Member
domenic commented Jan 23, 2017 edited

whatwg/url#205 is almost ready to go as a first pass. Still to-do is figuring out where to put the deploy.sh script, and changing the Makefile to allow local deploy. (The latter depends on the former.)

After that we can start using it on the simple repos (i.e. those that only have the main spec file and maybe .htaccess). Then we can start working on the stuff mentioned in the OP to allow it to be used in more repos (e.g. those containing other files).

@domenic domenic added a commit to whatwg/resources.whatwg.org that referenced this issue Jan 23, 2017
@domenic domenic Add common deploy script
Closes whatwg/meta#6 and closes whatwg/meta#9.
96643f5
@domenic domenic added a commit to whatwg/resources.whatwg.org that referenced this issue Jan 23, 2017
@domenic domenic Add common deploy script
Closes whatwg/meta#6 and closes whatwg/meta#9.
32fd0df
@domenic domenic referenced this issue in whatwg/resources.whatwg.org Jan 23, 2017
Merged

Add common deploy script #41

@domenic
Member
domenic commented Jan 23, 2017

OK, this should be ready for review:

I also included editorconfig and gitattributes for URL where I was there, in the spirit of minimizing the number of meta commits.

@annevk
Member
annevk commented Jan 24, 2017

I'm not sure who would be best to review since you're the person with the most knowledge of this. Maybe @sideshowbarker can look if the shell script looks reasonable?

I'll try to have a look anyway to see if I can spot anything.

@domenic domenic added a commit to whatwg/dom that referenced this issue Jan 24, 2017
@domenic domenic Meta: update spec deploy and editing infrastructure
- Updates to the deduplicated deploy script; part of whatwg/meta#6
- As such, stops deploying commit snapshots for branches; part of whatwg/meta#9
- Fixes .travis.yml syntax; part of whatwg/meta#4
- Adds .gitattributes and .editorconfig; part of whatwg/meta#7
68148f2
@domenic domenic added a commit to whatwg/resources.whatwg.org that referenced this issue Jan 24, 2017
@domenic domenic Add a common deploy script
The building block for solving whatwg/meta#6 and whatwg/meta#9.
585b6d9
@annevk annevk referenced this issue Jan 24, 2017
Open

Use generic deploy.sh everywhere #11

1 of 6 tasks complete
@domenic domenic added a commit to whatwg/dom that referenced this issue Jan 24, 2017
@domenic domenic Meta: update spec deploy and editing infrastructure
- Updates to the deduplicated deploy script; part of whatwg/meta#6
- As such, stops deploying commit snapshots for branches; part of whatwg/meta#9
- Fixes .travis.yml syntax; part of whatwg/meta#4
- Adds .gitattributes and .editorconfig; part of whatwg/meta#7
4c5c099
@sideshowbarker
Member

For the record, this all looks fine to me (sorry for not having responded here earlier).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment