MyPaint Wiki and DNS hosting to be closed down: migrate content, zones #242

Open
achadwick opened this Issue Mar 6, 2015 · 25 comments

Projects

None yet

6 participants

@achadwick
Member

Our hosting provider, Techmight, will be closing down the server used for the MyPaint wiki shortly, and we need to find a new home for the useful subset of the wiki content. We have no firm drop-dead date, but it has been approaching for a couple of months already, and I would like to get this sorted within the next two weeks (our existing hosting provider has been kind enough already, let's get moving)

This will not affect web hosting of the MyPaint home page or the forums because those are hosted by a different provider.

I've been discussing this migration over email with @jonnor, @martinxyz, and our current awesome hosting provider as represented by @sikhnerd who's been very helpful and tolerant of my laziness and ignorant questions so far.

Here's the rough plan nobody objected to, slightly expanded:

Migration plan

I would like to migrate some (but not all) of our wiki's content to Github. Since the rest of development activity has pretty much moved to github already, this would allow us to coordinate things better.

I'm (still) not that familiar with mediawiki hosting, and have always considered keeping the thing spam-free something of a chore. However it is used by our users (decreasingly of late) as a point for distribution of brush packs, and as a place for writing and maintaining user documentation. In particular, we link to the wiki from within the app to allow users to download brush packages, using the URL http://wiki.mypaint.info/index.php?title=Brush_Packages/redirect_mypaint_1.1_gui.

  • The brush pack .zip files are all hosted externally, although their preview images seem to be hosted on the wiki. A custom 404 page with a manual or JS redirect to a real page for the above case should work around a lack of PHP, allowing us to use any web host we like for the old brushpack download URL.
  • User documentation can and should be migrated piecemeal to another wiki, and perhaps the Brush_Packages page should live there too. I'm intending to move content to the main MyPaint repo wiki, https://github.com/mypaint/mypaint/wiki, for this: currently it's unused.
  • Those I think are the remaining interesting uses of the old wiki. I'm sure we'll find more.

(To be honest, brushpacks could be distributed quite sensibly as a separate github repository too. But that would require coordination with the authors, and discussion about licenses.)

I'm already taking steps to migrate DNS hosting for the mypaint.info and mypaint.org domains, and will thus be able to point them at some Github Pages hosting I guess. The docs say that custom domains are possible for these things.

Checklist

  1. Lock down the existing wiki
  2. Announce the move, making it clear that some data will disappear, so that users can retrieve any artwork that would otherwise be lost
  3. Back it up, both files and database
    • Copy the backup privately (@achadwick has it stashed and backed up)
    • Bring up on a private mediawiki instance (needed?)
  4. Begin writing pages & wiki transcriptions on github, preserve the content we need.
  5. Switch over the NS records for the domain in consultation with the existing registrar and hosting provider. @jonnor will retain ownership of the domain initially, and I'll be seeing how I get on with the DNS provider I've selected (mythic-beasts.com)
  6. Allow the old hosting to expire gracefully.
  7. Complete any late-stage migrations from those backups.
@achadwick achadwick added the project label Mar 6, 2015
@achadwick achadwick self-assigned this Mar 6, 2015
@achadwick achadwick added this to the MyPaint 1.2 milestone Mar 6, 2015
@achadwick
Member

My PGP key, if any passwords need to be communicated:

pub   4096R/2878B299 2013-01-01
      Key fingerprint = C023 91F4 BBA4 F0E2 B27C  6BFF 6E30 37E1 2878 B299
uid                  Andrew Chadwick <a.t.chadwick@gmail.com>
sub   4096R/B2A4FFD5 2013-01-01
@achadwick
Member

Big red message put up on the front page, and $wgReadOnly set. Both messages point here.

@odysseywestra
Member

I will be willing to help out with this. I was going to propose the wiki be
moved to github for documentation purposes, but you beat me to the punch.

With the brush packs, I would vote to encourage them to utilize github,
just so they could take advantage of the issues page.

If you at least want to extent the hosting so we have control of when the
wiki shuts down, we could set up temporary sever on Digital Ocean. Plus if
we a refer link we could use it for a couple of months without any cost.

On Fri, Mar 6, 2015, 10:52 AM Andrew Chadwick notifications@github.com
wrote:

Big red message put up on the front page, and $wgReadOnly set. Both
messages point here.


Reply to this email directly or view it on GitHub
#242 (comment).

@odysseywestra
Member

Plus with moving to github will lower the threshold for contributing, with
only a github account and knowledge of markdown.

On Fri, Mar 6, 2015, 11:03 AM Albert Westra odysseywestra@gmail.com wrote:

I will be willing to help out with this. I was going to propose the wiki
be moved to github for documentation purposes, but you beat me to the punch.

With the brush packs, I would vote to encourage them to utilize github,
just so they could take advantage of the issues page.

If you at least want to extent the hosting so we have control of when the
wiki shuts down, we could set up temporary sever on Digital Ocean. Plus if
we a refer link we could use it for a couple of months without any cost.

On Fri, Mar 6, 2015, 10:52 AM Andrew Chadwick notifications@github.com
wrote:

Big red message put up on the front page, and $wgReadOnly set. Both
messages point here.


Reply to this email directly or view it on GitHub
#242 (comment).

@achadwick
Member

Blog updated, twitters poked, notifications done.

I've asked people to come here and link to stuff they want migrating across. But once we've laid a few foundations, I guess people can pitch in (I think anyone with a github account can do that...)

@achadwick
Member

@odysseywestra I'm actually kind of OK with just manually migrating the useful data across. It forces us to look at it and decide if it's worth keeping. Extreme documenting, if you will.

(Though if anyone has a MediaWiki → Github-Flavoured-Markdown translator tool out there, that'd certainly help)

@SikhNerd
SikhNerd commented Mar 6, 2015

For automated mass-move using pandoc: https://github.com/philipashlock/mediawiki-to-markdown

@iirelu
Contributor
iirelu commented Mar 7, 2015

List of links to wiki.mypaint.info in Mypaint:

gui/drawwindow.py
61:BRUSHPACK_URI = 'http://wiki.mypaint.info/index.php?title=Brush_Packages/redirect_mypaint_1.1_gui'

gui/application.py
572:            #       http://wiki.mypaint.info/File:Pressure_workaround.png

README_OSX.md
8:http://wiki.mypaint.info/Development/Packaging#OSX. Feedback welcome.

README.md
18:  - [Wiki](http://wiki.mypaint.info/)
22:  - [Introductory docs for developers](http://wiki.mypaint.info/index.php?title=Documentation/ForDevelopers)

brushlib/README.md
40:http://wiki.mypaint.info/Brushlib

brushlib/mypaint-tiled-surface.c
324:    // http://wiki.mypaint.info/Development/Documentation/Brushlib

List of pages in the Mypaint wiki, so we can get a picture of what needs migrating. You can also use this while it's still up.

@achadwick
Member

Okay, the domain has recently vanished off the techmight DNS servers because techmight.com

$ host NS1.TECHMIGHT.COM
NS1.TECHMIGHT.COM has address 69.175.114.211
NS1.TECHMIGHT.COM mail is handled by 0 NS1.TECHMIGHT.COM.
$ host NS2.TECHMIGHT.COM
NS2.TECHMIGHT.COM has address 69.175.114.212
NS2.TECHMIGHT.COM mail is handled by 0 NS2.TECHMIGHT.COM.
$ dig -t ns mypaint.info @ns1.techmmight.com
dig: couldn't get address for 'ns1.techmmight.com': not found
$ dig -t ns mypaint.org @ns2.techmmight.com
dig: couldn't get address for 'ns2.techmmight.com': not found
$ dig -t ns mypaint.info @69.175.114.211
; <<>> DiG 9.9.5-9-Debian <<>> -t ns mypaint.info @69.175.114.211
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: SERVFAIL, id: 50639
;; flags: qr rd; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 1
;; WARNING: recursion requested but not available

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;mypaint.info.          IN  NS

;; Query time: 155 msec
;; SERVER: 69.175.114.211#53(69.175.114.211)
;; WHEN: Mon Mar 09 22:52:02 GMT 2015
;; MSG SIZE  rcvd: 41

$ 
[similarly for ns2]

That's not helpful, and it's hampering our migration. @SikhNerd could you look into this? There's clearly been some miscommunication here over timescales.


@jonnor Please can you point the NS records for the two domains ASAP at {ns1,ns2}.mythic-beasts.com, using

Name IPv4 address IPv6 address
ns1.mythic-beasts.com 69.56.173.190 2600:3c00::f03c:91ff:fe96:beac
ns2.mythic-beasts.com 93.93.128.67 2a00:1098:0:80:1000::10

-- https://www.mythic-beasts.com/support/domains/nameservers

for the glue records, and I'll do the rest as best I can.

@SikhNerd
SikhNerd commented Mar 9, 2015

Don't know how this happened, but there is no confusion of timescales and this is unintentional. Please review the dns zones per my email.

@achadwick
Member

@SikhNerd Thanks for the prompt action and the reassurances. {ns1,ns2}.techmight.com are authoritative for the two domains again, and I see RRs for wiki and www with dig (when I'm driving properly). Wiki and zone are available, and I can get busy again.

@jonnor You might want to hang back on that slightly panicked request above!

@odysseywestra
Member

So how do we want to structure the Wiki in github since we can clone it and work on it offline?

@achadwick
Member

@odysseywestra Flat is best for this wiki. Separate [EDIT: words in file names] with hyphens. If you start putting things in subdirs, [[links]] only [EDIT: work] with the basename (the bit after the last /).

@odysseywestra
Member

So kinda like how you would structure a website? I was thinking about separate forlder for the images like in /img for starters. Probably make separte forlders for each page, but that may make it too complicated.

Plus do we want to create a separate wiki for libmypaint and just link mypaint wiki to it or would you rather just keep them as one and just link it in libmypaint's readme.md file? Me personaly I would go with the latter.

@achadwick
Member

DNS and redirects bought and in place on mythic-beasts and tested, to the extent I can without the domain switchover.

There's a slight wrinkle in the redirect for the wiki which I've asked Support there to look at (basically the "get more brushes" link will break on old MyPaint releases until it's fixed), but other than that it looks good!

NS+glues switchover is scheduled for Monday 16 via @jonnor and I think we're ready to go there. There are more pages to port over, but I have a full dump of the old wiki site and will be migrating pages over as necessary. There aren't that many really...!

@achadwick
Member

@odysseywestra you can, but I've chosen to keep the filenames of images prefixed by their page filename. They have to be unique with this wiki, and there's not much sharing of images between pages.

@achadwick
Member

Redirect wrinkles should all be sorted out, meaning all we need to do is switch over the nameservers to be m-b's and not techmight's. And migrate over the remaining pages...

@achadwick achadwick added the docs label Mar 23, 2015
@achadwick achadwick added a commit that referenced this issue Mar 28, 2015
@achadwick achadwick Remove+replace links to old wiki 4c30559
@achadwick
Member

We have a drop-dead date from our old hosting provider of May 1st, 2015.

As far as I am concerned, we are completely ready to have the github wiki take over wiki provision (~ some tidyups that can happen later), we're completely ready with the www. redirect, and are now just waiting for the NS switchover.

@jonnor - please can you make arrangements with the existing registrar for the two domains? Many thanks.

@jonnor
Member
jonnor commented Apr 19, 2015

@achadwick mypaint.org has now been switched over to ns{1,2}.mythic-beasts.com
mypaint.info is on another account, still working on that one.

@jonnor
Member
jonnor commented Apr 19, 2015

mypaint.info also moved to ns{1,2}.mythic-beasts.com

Will be some hours propagation time before changes are in full effect

@achadwick
Member

Thanks for that, Jon. New redirects are looking good from here.

@SikhNerd we can now let the old provision expire gracefully. Nothing more needed from us for the old servers. Many thanks for hosting these domains and the old wiki for us for so long.

@achadwick
Member

Moving forwards, the new wiki is up and open for edits. And please do! I believe all you need to edit it is a Github account.

@achadwick
Member

All that remains is:

[ ] Complete any late-stage migrations from those backups.

which is now my task. There's not that much remaining to do.

@odysseywestra
Member

Is this considered done now?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment