Migrate help.openstreetmap.org from OSQA

[simonpoole]

ASKBOT is an actively maintained fork of qsqa (that was last worked on something like 5 years ago), see:
https://github.com/ASKBOT/askbot-devel and https://askbot.com/

If we want to continue to run our own Q&A site migrating would make sense and would likely address many of the issues we have with QSQA (not just that it is abandonware). The alternative would be to give up running our own system and move to stackexchange, which has IMHO some downsides including loosing the content I believe.

Other issues to consider:

• while ASKBOT still works with mysql, they support postgres better, I assume that is OK with us in any case?
• there is, for unfathomable reasons, no off the shelf migration script/facility, this implies that some coding is likely to be involved. Potentially however the people as ASKBOT would be willing to do this for a reasonably small some of money and that would help everybody using QSQA..

Comments @woodpeck @systemed and anybody else answering questions on help?

[tomhughes]

If it's a fork I'm surprised the database can't just be migrated or have their been schema changes on the OSQA side since the fork?

[simonpoole]

The later probably not, and I wouldn't rule out that we could migrate the mysql DB and then migrate to Postgres. From reading the support forums it is not quite clear what the issue is, but I'll dig a bit more.

This http://askbot.org/doc/upgrade.html may be part of the problem.

[EdLoach]

Sounds sensible to move to something that is actively maintained if it isn't too difficult to do.

[woodpeck]

OSQA works well enough for me, the glitches that are there don't bother me. I have never used Askbot, it doesn't exactly look like there's a huge developer community behind it apart from the company who runs it - will anything remain if their business tanks? I don't mind staying with OSQA but I don't mind switching either. If that should mean losing our content I think that wouldn't even be too bad, many the answers in OSQA are outdated. We count pull a read-only static image from OSQA for reference purposes and copy over relevant stuff when a question pops up... plus a reboot would create a fair karma playing field where you don't have this woodpeck guy with twice as many karma points as the next contender.

[tomhughes]

@woodpeck problem is that I need to upgrade that machine to 16.04 at some point but I have no confidence OSQA won't break (it did when we went to 14.04 and we had to patch it).

[SomeoneElseOSM]

If it makes sense to move (and I can see why there would, given the issues we had and to some extent still have), I don't see a particular problem with doing so.

There are some "old answers" that are worth preserving, but bluntly some sort of static HTML would work for those. Many more old answers "were correct ages ago" but now no longer are.

I don't think that a "flattening" of user reputation would be a problem either.

[SomeoneElseOSM]

I've been asked to add a link to https://wiki.openstreetmap.org/wiki/Talk:Help.openstreetmap.org#Migration_QSQA_-.3E_ASKBOT here, but the content of the message is small enough to post in full:

"Sounds quite good for me. However, if we are migrating to anywhere we rather should think about how to solve the issue of outdated answers. Just deleting the whole thing every 7 years is a waste. There ARE many still good answers (yes, also much irrelevant or even wrong answers nowadays) and I try to take care of them (updating, organizing) as good as possible. I also would like to have incoming link not to be broken if possible (yes, setting it up as static html clone might work). Regarding the "outdated" problem, also see the section above. Regarding possible Karma flattening: that is likely no problem. --Aseerel4c26 (talk) 20:04, 21 February 2017 (UTC)"

[scaidermern]

Keeping good questions/answers is certainly a good idea. We should make sure people are still able to find them via search engines.

[woodpeck]

Assuming for a moment that copying over part but not all of existing Qs&As is possible. perhaps we could share the load of judging which questions/answers to keep and which to kill by resetting all scores to 0, and then asking everyone to look through stuff and upvote what they think should be kept...

[scaidermern]

Yes. Maybe we can go through the top 50 questions here and decide which ones to keep: https://help.openstreetmap.org/questions/?sort=mostvoted

If they can't get migrated maybe we could still copy'n'pasting the question and 1-2 good answers and then mark them as community wiki.

[pnorman]

I think moving off of qsqa is the only reasonable option given it's unmaintained nature and problems that have required patching, so it's only a question of if askbot the best option.

There is information on migrating, so it's just a question of how far back the migrations go.

[grischard]

The Wireshark project solved this by simply archiving their osmqa site.

[tomhughes]

The problem is that I need to upgrade the machine and doing so will almost certainly break OSQA meaning that even running it in RO mode is likely to be hard.

[simonpoole]

@tomhughes @gravitystorm @zerebubuth this would seem like a fairly self-contained activity that could be farmed out to an interested person from the wider community that wants to help. Again objections to raising this on the a suitable mailing list (I'm pushing a bit because besides the discussed issues, we should be fixing the gravatar support in some way from a data protection pov, and it doesn't make sense to do this in the old unsupported code).

[tomhughes]

If you can find somebody that can figure out how to migrate the data then sure.

I actually have a chef branch where I started work on code to install and manage askbot.

[RobJN]

Is this any help?
https://gist.github.com/cholin/171898332d63afb8371c

[nebulon42]

How big is the MySQL database? Can one get access to a dump?

[tomhughes]

It's postgres, not mysql. Ignoring the (vast) session table it looks like there is a little over 100Mb in the tables.

If you're volunteering to attempt a conversion then we could probably come to an arrangement to make it available, subject to appropriate agreements around safeguarding any private data.

[nebulon42]

I know I shouldn't because I have too many projects already, but somehow this interests me. :)
So let's say I volunteer to attempt a conversion without any commitment to an ETA. But without trying it's all theory and if not successful there might be at least lessons learned for the next attempt.
I'm fine with setting up a NDA or something similar. Being part of CWG might help already though there are no written agreements of course.

[slachiewicz]

Maybe we can try with Discourse - more active development also Postgress and Ruby based.
https://github.com/discourse/discourse/blob/master/docs/INSTALL.md

[ppKrauss]

Hi, about @slachiewicz's suggestion,

Maybe we can try with Discourse ...

See meta.discourse/porting-from-askbot-to-discourse ... Seems that the major investment for us is to enhance this embrionary adaptation.

[tomhughes]

Discourse is a forum (or a replacement for forums) not a Q+A site.

That is kind of evident in the fact that it comes from one of the founders of Stack Overflow and was specifically designed to address a different use case...

[slachiewicz]

As you can see @ppKrauss in the repository, you can boast a large and active number of migration scripts https://github.com/discourse/discourse/tree/master/script/import_scripts

[simonpoole]

@ppKrauss @slachiewicz pls no. While discourse may have its merits as a forum replacement (just that I've never found any) it is really completely at odds with what the help site is supposed to be doing (as @tomhughes has said).

[ppKrauss]

Some news about our options: meta.discourse community's positions and suggestions.

[pnorman]

So, I had a look at AskBot looking at testing a conversion.

I can't recommend it because we'll have the same problems as now.

All versions of AskBot rely on Django versions that were released 3-4 years ago and are EOL. The commit activity on its repository is minimal.

Keeping OSQA on a machine running 16.04 14.04! is not a long-term option. Running Django 1.8 which has stopped getting security patches is also not a long-term solution.

[slachiewicz]

As workaround maybe pack all Django OSQA stuff to Docker?

[pnorman]

As workaround maybe pack all Django OSQA stuff to Docker?

That would still be running 14.04 for everything except the kernel, so there'd be the same problems.

Since 14.04 hits EOL in April, we need to upgrade the machine by then, don't we?

Should we set sometime in March as a deadline for we're going to upgrade the machine, even if it causes problems with help.osm.org?

[pnorman]

I looked at the options listed on a SE question and these are what I see as an active alternative

• question2answer code
• scoold code (Might not be fully open source)

Other options that are not open-source are

• AnswerHub, by the people who wrote OSQA.
• Stack Exchange Enterprise
• Try to get a Stack Exchange created for OSM
• Drop all OSMF support for a Q&A site. Stack Exchange might fill the gap, or might not

Nothing stands out as the right option that is better than all alternatives.

[simonpoole]

We could naturally pay the askbot developer to migrate askbot to a current django version (this would naturally only solve one of many issues).

[lonvia]

Instead of using a full Docker container, the current OSQA can be simply run in a Python virtualenv with the older python libraries. Doesn't solve any of the major issue either but at least gives an upgrade path for the machine.

[rory]

Another reason to switch from OSQA: It doesn't support all OSM usernames. I cannot log into help.openstreetmap.org with my OSM account (rorym 🏳️‍🌈).

[tomhughes]

Could we please keep this vaguely on the subject @rory - that is nothing to do with OSQA in any case and everything to do with our custom authentication plugin and my non-existent python skills.

[EdLoach]

Looking at pnorman's comment and in particular question2answer I did find this migration script from OSQA in case it proves to be useful, or at least a start.