New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

MyPaint Wiki and DNS hosting to be closed down: migrate content, zones #242

Closed
achadwick opened this Issue Mar 6, 2015 · 26 comments

Comments

Projects
None yet
6 participants
@achadwick
Member

achadwick commented Mar 6, 2015

Our hosting provider, Techmight, will be closing down the server used for the MyPaint wiki shortly, and we need to find a new home for the useful subset of the wiki content. We have no firm drop-dead date, but it has been approaching for a couple of months already, and I would like to get this sorted within the next two weeks (our existing hosting provider has been kind enough already, let's get moving)

This will not affect web hosting of the MyPaint home page or the forums because those are hosted by a different provider.

I've been discussing this migration over email with @jonnor, @martinxyz, and our current awesome hosting provider as represented by @SikhNerd who's been very helpful and tolerant of my laziness and ignorant questions so far.

Here's the rough plan nobody objected to, slightly expanded:

Migration plan

I would like to migrate some (but not all) of our wiki's content to Github. Since the rest of development activity has pretty much moved to github already, this would allow us to coordinate things better.

I'm (still) not that familiar with mediawiki hosting, and have always considered keeping the thing spam-free something of a chore. However it is used by our users (decreasingly of late) as a point for distribution of brush packs, and as a place for writing and maintaining user documentation. In particular, we link to the wiki from within the app to allow users to download brush packages, using the URL http://wiki.mypaint.info/index.php?title=Brush_Packages/redirect_mypaint_1.1_gui.

  • The brush pack .zip files are all hosted externally, although their preview images seem to be hosted on the wiki. A custom 404 page with a manual or JS redirect to a real page for the above case should work around a lack of PHP, allowing us to use any web host we like for the old brushpack download URL.
  • User documentation can and should be migrated piecemeal to another wiki, and perhaps the Brush_Packages page should live there too. I'm intending to move content to the main MyPaint repo wiki, https://github.com/mypaint/mypaint/wiki, for this: currently it's unused.
  • Those I think are the remaining interesting uses of the old wiki. I'm sure we'll find more.

(To be honest, brushpacks could be distributed quite sensibly as a separate github repository too. But that would require coordination with the authors, and discussion about licenses.)

I'm already taking steps to migrate DNS hosting for the mypaint.info and mypaint.org domains, and will thus be able to point them at some Github Pages hosting I guess. The docs say that custom domains are possible for these things.

Checklist

  1. Lock down the existing wiki
  2. Announce the move, making it clear that some data will disappear, so that users can retrieve any artwork that would otherwise be lost
  3. Back it up, both files and database
    • Copy the backup privately (@achadwick has it stashed and backed up)
    • Bring up on a private mediawiki instance (needed?)
  4. Begin writing pages & wiki transcriptions on github, preserve the content we need.
  5. Switch over the NS records for the domain in consultation with the existing registrar and hosting provider. @jonnor will retain ownership of the domain initially, and I'll be seeing how I get on with the DNS provider I've selected (mythic-beasts.com)
  6. Allow the old hosting to expire gracefully.
  7. Complete any late-stage migrations from those backups.

@achadwick achadwick added the project label Mar 6, 2015

@achadwick achadwick self-assigned this Mar 6, 2015

@achadwick achadwick added this to the MyPaint 1.2 milestone Mar 6, 2015

@achadwick

This comment has been minimized.

Show comment
Hide comment
@achadwick

achadwick Mar 6, 2015

Member

My PGP key, if any passwords need to be communicated:

pub   4096R/2878B299 2013-01-01
      Key fingerprint = C023 91F4 BBA4 F0E2 B27C  6BFF 6E30 37E1 2878 B299
uid                  Andrew Chadwick <a.t.chadwick@gmail.com>
sub   4096R/B2A4FFD5 2013-01-01
Member

achadwick commented Mar 6, 2015

My PGP key, if any passwords need to be communicated:

pub   4096R/2878B299 2013-01-01
      Key fingerprint = C023 91F4 BBA4 F0E2 B27C  6BFF 6E30 37E1 2878 B299
uid                  Andrew Chadwick <a.t.chadwick@gmail.com>
sub   4096R/B2A4FFD5 2013-01-01
@achadwick

This comment has been minimized.

Show comment
Hide comment
@achadwick

achadwick Mar 6, 2015

Member

Big red message put up on the front page, and $wgReadOnly set. Both messages point here.

Member

achadwick commented Mar 6, 2015

Big red message put up on the front page, and $wgReadOnly set. Both messages point here.

@odysseywestra

This comment has been minimized.

Show comment
Hide comment
@odysseywestra

odysseywestra Mar 6, 2015

Member

I will be willing to help out with this. I was going to propose the wiki be
moved to github for documentation purposes, but you beat me to the punch.

With the brush packs, I would vote to encourage them to utilize github,
just so they could take advantage of the issues page.

If you at least want to extent the hosting so we have control of when the
wiki shuts down, we could set up temporary sever on Digital Ocean. Plus if
we a refer link we could use it for a couple of months without any cost.

On Fri, Mar 6, 2015, 10:52 AM Andrew Chadwick notifications@github.com
wrote:

Big red message put up on the front page, and $wgReadOnly set. Both
messages point here.


Reply to this email directly or view it on GitHub
#242 (comment).

Member

odysseywestra commented Mar 6, 2015

I will be willing to help out with this. I was going to propose the wiki be
moved to github for documentation purposes, but you beat me to the punch.

With the brush packs, I would vote to encourage them to utilize github,
just so they could take advantage of the issues page.

If you at least want to extent the hosting so we have control of when the
wiki shuts down, we could set up temporary sever on Digital Ocean. Plus if
we a refer link we could use it for a couple of months without any cost.

On Fri, Mar 6, 2015, 10:52 AM Andrew Chadwick notifications@github.com
wrote:

Big red message put up on the front page, and $wgReadOnly set. Both
messages point here.


Reply to this email directly or view it on GitHub
#242 (comment).

@odysseywestra

This comment has been minimized.

Show comment
Hide comment
@odysseywestra

odysseywestra Mar 6, 2015

Member

Plus with moving to github will lower the threshold for contributing, with
only a github account and knowledge of markdown.

On Fri, Mar 6, 2015, 11:03 AM Albert Westra odysseywestra@gmail.com wrote:

I will be willing to help out with this. I was going to propose the wiki
be moved to github for documentation purposes, but you beat me to the punch.

With the brush packs, I would vote to encourage them to utilize github,
just so they could take advantage of the issues page.

If you at least want to extent the hosting so we have control of when the
wiki shuts down, we could set up temporary sever on Digital Ocean. Plus if
we a refer link we could use it for a couple of months without any cost.

On Fri, Mar 6, 2015, 10:52 AM Andrew Chadwick notifications@github.com
wrote:

Big red message put up on the front page, and $wgReadOnly set. Both
messages point here.


Reply to this email directly or view it on GitHub
#242 (comment).

Member

odysseywestra commented Mar 6, 2015

Plus with moving to github will lower the threshold for contributing, with
only a github account and knowledge of markdown.

On Fri, Mar 6, 2015, 11:03 AM Albert Westra odysseywestra@gmail.com wrote:

I will be willing to help out with this. I was going to propose the wiki
be moved to github for documentation purposes, but you beat me to the punch.

With the brush packs, I would vote to encourage them to utilize github,
just so they could take advantage of the issues page.

If you at least want to extent the hosting so we have control of when the
wiki shuts down, we could set up temporary sever on Digital Ocean. Plus if
we a refer link we could use it for a couple of months without any cost.

On Fri, Mar 6, 2015, 10:52 AM Andrew Chadwick notifications@github.com
wrote:

Big red message put up on the front page, and $wgReadOnly set. Both
messages point here.


Reply to this email directly or view it on GitHub
#242 (comment).

@achadwick

This comment has been minimized.

Show comment
Hide comment
@achadwick

achadwick Mar 6, 2015

Member

Blog updated, twitters poked, notifications done.

I've asked people to come here and link to stuff they want migrating across. But once we've laid a few foundations, I guess people can pitch in (I think anyone with a github account can do that...)

Member

achadwick commented Mar 6, 2015

Blog updated, twitters poked, notifications done.

I've asked people to come here and link to stuff they want migrating across. But once we've laid a few foundations, I guess people can pitch in (I think anyone with a github account can do that...)

@achadwick

This comment has been minimized.

Show comment
Hide comment
@achadwick

achadwick Mar 6, 2015

Member

@odysseywestra I'm actually kind of OK with just manually migrating the useful data across. It forces us to look at it and decide if it's worth keeping. Extreme documenting, if you will.

(Though if anyone has a MediaWiki → Github-Flavoured-Markdown translator tool out there, that'd certainly help)

Member

achadwick commented Mar 6, 2015

@odysseywestra I'm actually kind of OK with just manually migrating the useful data across. It forces us to look at it and decide if it's worth keeping. Extreme documenting, if you will.

(Though if anyone has a MediaWiki → Github-Flavoured-Markdown translator tool out there, that'd certainly help)

@jy2wong

This comment has been minimized.

Show comment
Hide comment
@jy2wong
Contributor

jy2wong commented Mar 6, 2015

@SikhNerd

This comment has been minimized.

Show comment
Hide comment
@SikhNerd

SikhNerd commented Mar 6, 2015

For automated mass-move using pandoc: https://github.com/philipashlock/mediawiki-to-markdown

@iirelu

This comment has been minimized.

Show comment
Hide comment
@iirelu

iirelu Mar 7, 2015

Contributor

List of links to wiki.mypaint.info in Mypaint:

gui/drawwindow.py
61:BRUSHPACK_URI = 'http://wiki.mypaint.info/index.php?title=Brush_Packages/redirect_mypaint_1.1_gui'

gui/application.py
572:            #       http://wiki.mypaint.info/File:Pressure_workaround.png

README_OSX.md
8:http://wiki.mypaint.info/Development/Packaging#OSX. Feedback welcome.

README.md
18:  - [Wiki](http://wiki.mypaint.info/)
22:  - [Introductory docs for developers](http://wiki.mypaint.info/index.php?title=Documentation/ForDevelopers)

brushlib/README.md
40:http://wiki.mypaint.info/Brushlib

brushlib/mypaint-tiled-surface.c
324:    // http://wiki.mypaint.info/Development/Documentation/Brushlib

List of pages in the Mypaint wiki, so we can get a picture of what needs migrating. You can also use this while it's still up.

Contributor

iirelu commented Mar 7, 2015

List of links to wiki.mypaint.info in Mypaint:

gui/drawwindow.py
61:BRUSHPACK_URI = 'http://wiki.mypaint.info/index.php?title=Brush_Packages/redirect_mypaint_1.1_gui'

gui/application.py
572:            #       http://wiki.mypaint.info/File:Pressure_workaround.png

README_OSX.md
8:http://wiki.mypaint.info/Development/Packaging#OSX. Feedback welcome.

README.md
18:  - [Wiki](http://wiki.mypaint.info/)
22:  - [Introductory docs for developers](http://wiki.mypaint.info/index.php?title=Documentation/ForDevelopers)

brushlib/README.md
40:http://wiki.mypaint.info/Brushlib

brushlib/mypaint-tiled-surface.c
324:    // http://wiki.mypaint.info/Development/Documentation/Brushlib

List of pages in the Mypaint wiki, so we can get a picture of what needs migrating. You can also use this while it's still up.

@achadwick

This comment has been minimized.

Show comment
Hide comment
@achadwick

achadwick Mar 9, 2015

Member

Okay, the domain has recently vanished off the techmight DNS servers because techmight.com

$ host NS1.TECHMIGHT.COM
NS1.TECHMIGHT.COM has address 69.175.114.211
NS1.TECHMIGHT.COM mail is handled by 0 NS1.TECHMIGHT.COM.
$ host NS2.TECHMIGHT.COM
NS2.TECHMIGHT.COM has address 69.175.114.212
NS2.TECHMIGHT.COM mail is handled by 0 NS2.TECHMIGHT.COM.
$ dig -t ns mypaint.info @ns1.techmmight.com
dig: couldn't get address for 'ns1.techmmight.com': not found
$ dig -t ns mypaint.org @ns2.techmmight.com
dig: couldn't get address for 'ns2.techmmight.com': not found
$ dig -t ns mypaint.info @69.175.114.211
; <<>> DiG 9.9.5-9-Debian <<>> -t ns mypaint.info @69.175.114.211
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: SERVFAIL, id: 50639
;; flags: qr rd; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 1
;; WARNING: recursion requested but not available

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;mypaint.info.          IN  NS

;; Query time: 155 msec
;; SERVER: 69.175.114.211#53(69.175.114.211)
;; WHEN: Mon Mar 09 22:52:02 GMT 2015
;; MSG SIZE  rcvd: 41

$ 
[similarly for ns2]

That's not helpful, and it's hampering our migration. @SikhNerd could you look into this? There's clearly been some miscommunication here over timescales.


@jonnor Please can you point the NS records for the two domains ASAP at {ns1,ns2}.mythic-beasts.com, using

Name IPv4 address IPv6 address
ns1.mythic-beasts.com 69.56.173.190 2600:3c00::f03c:91ff:fe96:beac
ns2.mythic-beasts.com 93.93.128.67 2a00:1098:0:80:1000::10

-- https://www.mythic-beasts.com/support/domains/nameservers

for the glue records, and I'll do the rest as best I can.

Member

achadwick commented Mar 9, 2015

Okay, the domain has recently vanished off the techmight DNS servers because techmight.com

$ host NS1.TECHMIGHT.COM
NS1.TECHMIGHT.COM has address 69.175.114.211
NS1.TECHMIGHT.COM mail is handled by 0 NS1.TECHMIGHT.COM.
$ host NS2.TECHMIGHT.COM
NS2.TECHMIGHT.COM has address 69.175.114.212
NS2.TECHMIGHT.COM mail is handled by 0 NS2.TECHMIGHT.COM.
$ dig -t ns mypaint.info @ns1.techmmight.com
dig: couldn't get address for 'ns1.techmmight.com': not found
$ dig -t ns mypaint.org @ns2.techmmight.com
dig: couldn't get address for 'ns2.techmmight.com': not found
$ dig -t ns mypaint.info @69.175.114.211
; <<>> DiG 9.9.5-9-Debian <<>> -t ns mypaint.info @69.175.114.211
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: SERVFAIL, id: 50639
;; flags: qr rd; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 1
;; WARNING: recursion requested but not available

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;mypaint.info.          IN  NS

;; Query time: 155 msec
;; SERVER: 69.175.114.211#53(69.175.114.211)
;; WHEN: Mon Mar 09 22:52:02 GMT 2015
;; MSG SIZE  rcvd: 41

$ 
[similarly for ns2]

That's not helpful, and it's hampering our migration. @SikhNerd could you look into this? There's clearly been some miscommunication here over timescales.


@jonnor Please can you point the NS records for the two domains ASAP at {ns1,ns2}.mythic-beasts.com, using

Name IPv4 address IPv6 address
ns1.mythic-beasts.com 69.56.173.190 2600:3c00::f03c:91ff:fe96:beac
ns2.mythic-beasts.com 93.93.128.67 2a00:1098:0:80:1000::10

-- https://www.mythic-beasts.com/support/domains/nameservers

for the glue records, and I'll do the rest as best I can.

@SikhNerd

This comment has been minimized.

Show comment
Hide comment
@SikhNerd

SikhNerd Mar 9, 2015

Don't know how this happened, but there is no confusion of timescales and this is unintentional. Please review the dns zones per my email.

SikhNerd commented Mar 9, 2015

Don't know how this happened, but there is no confusion of timescales and this is unintentional. Please review the dns zones per my email.

@achadwick

This comment has been minimized.

Show comment
Hide comment
@achadwick

achadwick Mar 9, 2015

Member

@SikhNerd Thanks for the prompt action and the reassurances. {ns1,ns2}.techmight.com are authoritative for the two domains again, and I see RRs for wiki and www with dig (when I'm driving properly). Wiki and zone are available, and I can get busy again.

@jonnor You might want to hang back on that slightly panicked request above!

Member

achadwick commented Mar 9, 2015

@SikhNerd Thanks for the prompt action and the reassurances. {ns1,ns2}.techmight.com are authoritative for the two domains again, and I see RRs for wiki and www with dig (when I'm driving properly). Wiki and zone are available, and I can get busy again.

@jonnor You might want to hang back on that slightly panicked request above!

@odysseywestra

This comment has been minimized.

Show comment
Hide comment
@odysseywestra

odysseywestra Mar 14, 2015

Member

So how do we want to structure the Wiki in github since we can clone it and work on it offline?

Member

odysseywestra commented Mar 14, 2015

So how do we want to structure the Wiki in github since we can clone it and work on it offline?

@achadwick

This comment has been minimized.

Show comment
Hide comment
@achadwick

achadwick Mar 15, 2015

Member

@odysseywestra Flat is best for this wiki. Separate [EDIT: words in file names] with hyphens. If you start putting things in subdirs, [[links]] only [EDIT: work] with the basename (the bit after the last /).

Member

achadwick commented Mar 15, 2015

@odysseywestra Flat is best for this wiki. Separate [EDIT: words in file names] with hyphens. If you start putting things in subdirs, [[links]] only [EDIT: work] with the basename (the bit after the last /).

@odysseywestra

This comment has been minimized.

Show comment
Hide comment
@odysseywestra

odysseywestra Mar 15, 2015

Member

So kinda like how you would structure a website? I was thinking about separate forlder for the images like in /img for starters. Probably make separte forlders for each page, but that may make it too complicated.

Plus do we want to create a separate wiki for libmypaint and just link mypaint wiki to it or would you rather just keep them as one and just link it in libmypaint's readme.md file? Me personaly I would go with the latter.

Member

odysseywestra commented Mar 15, 2015

So kinda like how you would structure a website? I was thinking about separate forlder for the images like in /img for starters. Probably make separte forlders for each page, but that may make it too complicated.

Plus do we want to create a separate wiki for libmypaint and just link mypaint wiki to it or would you rather just keep them as one and just link it in libmypaint's readme.md file? Me personaly I would go with the latter.

@achadwick

This comment has been minimized.

Show comment
Hide comment
@achadwick

achadwick Mar 15, 2015

Member

DNS and redirects bought and in place on mythic-beasts and tested, to the extent I can without the domain switchover.

There's a slight wrinkle in the redirect for the wiki which I've asked Support there to look at (basically the "get more brushes" link will break on old MyPaint releases until it's fixed), but other than that it looks good!

NS+glues switchover is scheduled for Monday 16 via @jonnor and I think we're ready to go there. There are more pages to port over, but I have a full dump of the old wiki site and will be migrating pages over as necessary. There aren't that many really...!

Member

achadwick commented Mar 15, 2015

DNS and redirects bought and in place on mythic-beasts and tested, to the extent I can without the domain switchover.

There's a slight wrinkle in the redirect for the wiki which I've asked Support there to look at (basically the "get more brushes" link will break on old MyPaint releases until it's fixed), but other than that it looks good!

NS+glues switchover is scheduled for Monday 16 via @jonnor and I think we're ready to go there. There are more pages to port over, but I have a full dump of the old wiki site and will be migrating pages over as necessary. There aren't that many really...!

@achadwick

This comment has been minimized.

Show comment
Hide comment
@achadwick

achadwick Mar 15, 2015

Member

@odysseywestra you can, but I've chosen to keep the filenames of images prefixed by their page filename. They have to be unique with this wiki, and there's not much sharing of images between pages.

Member

achadwick commented Mar 15, 2015

@odysseywestra you can, but I've chosen to keep the filenames of images prefixed by their page filename. They have to be unique with this wiki, and there's not much sharing of images between pages.

@achadwick

This comment has been minimized.

Show comment
Hide comment
@achadwick

achadwick Mar 16, 2015

Member

Redirect wrinkles should all be sorted out, meaning all we need to do is switch over the nameservers to be m-b's and not techmight's. And migrate over the remaining pages...

Member

achadwick commented Mar 16, 2015

Redirect wrinkles should all be sorted out, meaning all we need to do is switch over the nameservers to be m-b's and not techmight's. And migrate over the remaining pages...

@achadwick

This comment has been minimized.

Show comment
Hide comment
@achadwick

achadwick Apr 19, 2015

Member

We have a drop-dead date from our old hosting provider of May 1st, 2015.

As far as I am concerned, we are completely ready to have the github wiki take over wiki provision (~ some tidyups that can happen later), we're completely ready with the www. redirect, and are now just waiting for the NS switchover.

@jonnor - please can you make arrangements with the existing registrar for the two domains? Many thanks.

Member

achadwick commented Apr 19, 2015

We have a drop-dead date from our old hosting provider of May 1st, 2015.

As far as I am concerned, we are completely ready to have the github wiki take over wiki provision (~ some tidyups that can happen later), we're completely ready with the www. redirect, and are now just waiting for the NS switchover.

@jonnor - please can you make arrangements with the existing registrar for the two domains? Many thanks.

@jonnor

This comment has been minimized.

Show comment
Hide comment
@jonnor

jonnor Apr 19, 2015

Member

@achadwick mypaint.org has now been switched over to ns{1,2}.mythic-beasts.com
mypaint.info is on another account, still working on that one.

Member

jonnor commented Apr 19, 2015

@achadwick mypaint.org has now been switched over to ns{1,2}.mythic-beasts.com
mypaint.info is on another account, still working on that one.

@jonnor

This comment has been minimized.

Show comment
Hide comment
@jonnor

jonnor Apr 19, 2015

Member

mypaint.info also moved to ns{1,2}.mythic-beasts.com

Will be some hours propagation time before changes are in full effect

Member

jonnor commented Apr 19, 2015

mypaint.info also moved to ns{1,2}.mythic-beasts.com

Will be some hours propagation time before changes are in full effect

@achadwick

This comment has been minimized.

Show comment
Hide comment
@achadwick

achadwick Apr 20, 2015

Member

Thanks for that, Jon. New redirects are looking good from here.

@SikhNerd we can now let the old provision expire gracefully. Nothing more needed from us for the old servers. Many thanks for hosting these domains and the old wiki for us for so long.

Member

achadwick commented Apr 20, 2015

Thanks for that, Jon. New redirects are looking good from here.

@SikhNerd we can now let the old provision expire gracefully. Nothing more needed from us for the old servers. Many thanks for hosting these domains and the old wiki for us for so long.

@achadwick

This comment has been minimized.

Show comment
Hide comment
@achadwick

achadwick Apr 20, 2015

Member

Moving forwards, the new wiki is up and open for edits. And please do! I believe all you need to edit it is a Github account.

Member

achadwick commented Apr 20, 2015

Moving forwards, the new wiki is up and open for edits. And please do! I believe all you need to edit it is a Github account.

@achadwick

This comment has been minimized.

Show comment
Hide comment
@achadwick

achadwick Apr 20, 2015

Member

All that remains is:

[ ] Complete any late-stage migrations from those backups.

which is now my task. There's not that much remaining to do.

Member

achadwick commented Apr 20, 2015

All that remains is:

[ ] Complete any late-stage migrations from those backups.

which is now my task. There's not that much remaining to do.

@achadwick achadwick modified the milestones: MyPaint 1.2.0 project management, MyPaint 1.1.1-alpha.<final> Apr 22, 2015

@odysseywestra

This comment has been minimized.

Show comment
Hide comment
@odysseywestra

odysseywestra Nov 7, 2015

Member

Is this considered done now?

Member

odysseywestra commented Nov 7, 2015

Is this considered done now?

@achadwick

This comment has been minimized.

Show comment
Hide comment
@achadwick

achadwick Jan 27, 2017

Member

Yes, I think we're done on this one.

Member

achadwick commented Jan 27, 2017

Yes, I think we're done on this one.

@achadwick achadwick closed this Jan 27, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment