Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Migrate help.openstreetmap.org from OSQA #149

Open
simonpoole opened this issue Feb 21, 2017 · 47 comments
Open

Migrate help.openstreetmap.org from OSQA #149

simonpoole opened this issue Feb 21, 2017 · 47 comments

Comments

@simonpoole
Copy link

@simonpoole simonpoole commented Feb 21, 2017

ASKBOT is an actively maintained fork of qsqa (that was last worked on something like 5 years ago), see:
https://github.com/ASKBOT/askbot-devel and https://askbot.com/

If we want to continue to run our own Q&A site migrating would make sense and would likely address many of the issues we have with QSQA (not just that it is abandonware). The alternative would be to give up running our own system and move to stackexchange, which has IMHO some downsides including loosing the content I believe.

Other issues to consider:

  • while ASKBOT still works with mysql, they support postgres better, I assume that is OK with us in any case?
  • there is, for unfathomable reasons, no off the shelf migration script/facility, this implies that some coding is likely to be involved. Potentially however the people as ASKBOT would be willing to do this for a reasonably small some of money and that would help everybody using QSQA..

Comments @woodpeck @systemed and anybody else answering questions on help?

@tomhughes
Copy link
Member

@tomhughes tomhughes commented Feb 21, 2017

If it's a fork I'm surprised the database can't just be migrated or have their been schema changes on the OSQA side since the fork?

@simonpoole
Copy link
Author

@simonpoole simonpoole commented Feb 21, 2017

The later probably not, and I wouldn't rule out that we could migrate the mysql DB and then migrate to Postgres. From reading the support forums it is not quite clear what the issue is, but I'll dig a bit more.

This http://askbot.org/doc/upgrade.html may be part of the problem.

@EdLoach
Copy link

@EdLoach EdLoach commented Feb 21, 2017

Sounds sensible to move to something that is actively maintained if it isn't too difficult to do.

@woodpeck
Copy link
Contributor

@woodpeck woodpeck commented Feb 21, 2017

OSQA works well enough for me, the glitches that are there don't bother me. I have never used Askbot, it doesn't exactly look like there's a huge developer community behind it apart from the company who runs it - will anything remain if their business tanks? I don't mind staying with OSQA but I don't mind switching either. If that should mean losing our content I think that wouldn't even be too bad, many the answers in OSQA are outdated. We count pull a read-only static image from OSQA for reference purposes and copy over relevant stuff when a question pops up... plus a reboot would create a fair karma playing field where you don't have this woodpeck guy with twice as many karma points as the next contender.

@tomhughes
Copy link
Member

@tomhughes tomhughes commented Feb 21, 2017

@woodpeck problem is that I need to upgrade that machine to 16.04 at some point but I have no confidence OSQA won't break (it did when we went to 14.04 and we had to patch it).

@SomeoneElseOSM
Copy link

@SomeoneElseOSM SomeoneElseOSM commented Feb 21, 2017

If it makes sense to move (and I can see why there would, given the issues we had and to some extent still have), I don't see a particular problem with doing so.

There are some "old answers" that are worth preserving, but bluntly some sort of static HTML would work for those. Many more old answers "were correct ages ago" but now no longer are.

I don't think that a "flattening" of user reputation would be a problem either.

@SomeoneElseOSM
Copy link

@SomeoneElseOSM SomeoneElseOSM commented Mar 2, 2017

I've been asked to add a link to https://wiki.openstreetmap.org/wiki/Talk:Help.openstreetmap.org#Migration_QSQA_-.3E_ASKBOT here, but the content of the message is small enough to post in full:

"Sounds quite good for me. However, if we are migrating to anywhere we rather should think about how to solve the issue of outdated answers. Just deleting the whole thing every 7 years is a waste. There ARE many still good answers (yes, also much irrelevant or even wrong answers nowadays) and I try to take care of them (updating, organizing) as good as possible. I also would like to have incoming link not to be broken if possible (yes, setting it up as static html clone might work). Regarding the "outdated" problem, also see the section above. Regarding possible Karma flattening: that is likely no problem. --Aseerel4c26 (talk) 20:04, 21 February 2017 (UTC)"

@scaidermern
Copy link

@scaidermern scaidermern commented Mar 2, 2017

Keeping good questions/answers is certainly a good idea. We should make sure people are still able to find them via search engines.

@woodpeck
Copy link
Contributor

@woodpeck woodpeck commented Mar 2, 2017

Assuming for a moment that copying over part but not all of existing Qs&As is possible. perhaps we could share the load of judging which questions/answers to keep and which to kill by resetting all scores to 0, and then asking everyone to look through stuff and upvote what they think should be kept...

@scaidermern
Copy link

@scaidermern scaidermern commented Mar 3, 2017

Yes. Maybe we can go through the top 50 questions here and decide which ones to keep: https://help.openstreetmap.org/questions/?sort=mostvoted

If they can't get migrated maybe we could still copy'n'pasting the question and 1-2 good answers and then mark them as community wiki.

@pnorman
Copy link
Collaborator

@pnorman pnorman commented Mar 3, 2017

I think moving off of qsqa is the only reasonable option given it's unmaintained nature and problems that have required patching, so it's only a question of if askbot the best option.

There is information on migrating, so it's just a question of how far back the migrations go.

@grischard
Copy link

@grischard grischard commented Feb 18, 2018

The Wireshark project solved this by simply archiving their osmqa site.

@tomhughes
Copy link
Member

@tomhughes tomhughes commented Feb 18, 2018

The problem is that I need to upgrade the machine and doing so will almost certainly break OSQA meaning that even running it in RO mode is likely to be hard.

@simonpoole
Copy link
Author

@simonpoole simonpoole commented Feb 18, 2018

@tomhughes @gravitystorm @zerebubuth this would seem like a fairly self-contained activity that could be farmed out to an interested person from the wider community that wants to help. Again objections to raising this on the a suitable mailing list (I'm pushing a bit because besides the discussed issues, we should be fixing the gravatar support in some way from a data protection pov, and it doesn't make sense to do this in the old unsupported code).

@tomhughes
Copy link
Member

@tomhughes tomhughes commented Feb 18, 2018

If you can find somebody that can figure out how to migrate the data then sure.

I actually have a chef branch where I started work on code to install and manage askbot.

@gravitystorm gravitystorm changed the title Migrate help.openstreetmap.org from QSQA to ASKBOT Migrate help.openstreetmap.org from OSQA to ASKBOT Feb 26, 2018
@gravitystorm gravitystorm mentioned this issue Feb 26, 2018
49 of 50 tasks complete
@nebulon42
Copy link

@nebulon42 nebulon42 commented Mar 20, 2018

How big is the MySQL database? Can one get access to a dump?

@tomhughes
Copy link
Member

@tomhughes tomhughes commented Mar 20, 2018

It's postgres, not mysql. Ignoring the (vast) session table it looks like there is a little over 100Mb in the tables.

If you're volunteering to attempt a conversion then we could probably come to an arrangement to make it available, subject to appropriate agreements around safeguarding any private data.

@nebulon42
Copy link

@nebulon42 nebulon42 commented Mar 21, 2018

I know I shouldn't because I have too many projects already, but somehow this interests me. :)
So let's say I volunteer to attempt a conversion without any commitment to an ETA. But without trying it's all theory and if not successful there might be at least lessons learned for the next attempt.
I'm fine with setting up a NDA or something similar. Being part of CWG might help already though there are no written agreements of course.

@slachiewicz
Copy link

@slachiewicz slachiewicz commented Jul 16, 2018

Maybe we can try with Discourse - more active development also Postgress and Ruby based.
https://github.com/discourse/discourse/blob/master/docs/INSTALL.md

@ppKrauss
Copy link

@ppKrauss ppKrauss commented Aug 10, 2018

Hi, about @slachiewicz's suggestion,

Maybe we can try with Discourse ...

See meta.discourse/porting-from-askbot-to-discourse ... Seems that the major investment for us is to enhance this embrionary adaptation.

@tomhughes
Copy link
Member

@tomhughes tomhughes commented Aug 10, 2018

Discourse is a forum (or a replacement for forums) not a Q+A site.

That is kind of evident in the fact that it comes from one of the founders of Stack Overflow and was specifically designed to address a different use case...

@slachiewicz
Copy link

@slachiewicz slachiewicz commented Aug 10, 2018

As you can see @ppKrauss in the repository, you can boast a large and active number of migration scripts https://github.com/discourse/discourse/tree/master/script/import_scripts

@simonpoole
Copy link
Author

@simonpoole simonpoole commented Aug 10, 2018

@ppKrauss @slachiewicz pls no. While discourse may have its merits as a forum replacement (just that I've never found any) it is really completely at odds with what the help site is supposed to be doing (as @tomhughes has said).

@ppKrauss
Copy link

@ppKrauss ppKrauss commented Aug 10, 2018

@pnorman
Copy link
Collaborator

@pnorman pnorman commented Jan 14, 2019

So, I had a look at AskBot looking at testing a conversion.

I can't recommend it because we'll have the same problems as now.

All versions of AskBot rely on Django versions that were released 3-4 years ago and are EOL. The commit activity on its repository is minimal.

Keeping OSQA on a machine running 16.04 14.04! is not a long-term option. Running Django 1.8 which has stopped getting security patches is also not a long-term solution.

@slachiewicz
Copy link

@slachiewicz slachiewicz commented Jan 14, 2019

As workaround maybe pack all Django OSQA stuff to Docker?

@pnorman
Copy link
Collaborator

@pnorman pnorman commented Jan 14, 2019

As workaround maybe pack all Django OSQA stuff to Docker?

That would still be running 14.04 for everything except the kernel, so there'd be the same problems.

Since 14.04 hits EOL in April, we need to upgrade the machine by then, don't we?

Should we set sometime in March as a deadline for we're going to upgrade the machine, even if it causes problems with help.osm.org?

@pnorman pnorman changed the title Migrate help.openstreetmap.org from OSQA to ASKBOT Migrate help.openstreetmap.org from OSQA Jan 14, 2019
@pnorman
Copy link
Collaborator

@pnorman pnorman commented Jan 15, 2019

I looked at the options listed on a SE question and these are what I see as an active alternative

Other options that are not open-source are

  • AnswerHub, by the people who wrote OSQA.
  • Stack Exchange Enterprise
  • Try to get a Stack Exchange created for OSM
  • Drop all OSMF support for a Q&A site. Stack Exchange might fill the gap, or might not

Nothing stands out as the right option that is better than all alternatives.

@simonpoole
Copy link
Author

@simonpoole simonpoole commented Jan 15, 2019

We could naturally pay the askbot developer to migrate askbot to a current django version (this would naturally only solve one of many issues).

@lonvia
Copy link

@lonvia lonvia commented Jan 15, 2019

Instead of using a full Docker container, the current OSQA can be simply run in a Python virtualenv with the older python libraries. Doesn't solve any of the major issue either but at least gives an upgrade path for the machine.

@rory
Copy link

@rory rory commented Jan 15, 2019

Another reason to switch from OSQA: It doesn't support all OSM usernames. I cannot log into help.openstreetmap.org with my OSM account (rorym 🏳️‍🌈).

@tomhughes
Copy link
Member

@tomhughes tomhughes commented Jan 15, 2019

Could we please keep this vaguely on the subject @rory - that is nothing to do with OSQA in any case and everything to do with our custom authentication plugin and my non-existent python skills.

@EdLoach
Copy link

@EdLoach EdLoach commented Jan 15, 2019

Looking at pnorman's comment and in particular question2answer I did find this migration script from OSQA in case it proves to be useful, or at least a start.

@pnorman pnorman pinned this issue Feb 5, 2019
@ppKrauss
Copy link

@ppKrauss ppKrauss commented Mar 2, 2019

Discourse seems the only active and secure for a long-term investment.

Please, who likes the idea of migrating to "Discourse + Q&A" (there are two customizable options, Solved and Voting) click here "like"...

If 3 or more people like it, we can study how to adapt and migrate, then, after study, come back here with a solution proposal.


PS: Discourse can be used also as commom software infrastructure for other chanels, like the forum (see example in other multi-community).

@thomersch
Copy link
Member

@thomersch thomersch commented Oct 31, 2019

I briefly talked with the Askbot developer earlier this year, it looks like there is some activity. On the 0.11 branch they have a Django 2.2 compatible version, which is compatible with Python 3.5/3.6/3.7. They also have a Dockerfile nowadays which could help with isolating the installation.

@simonpoole
Copy link
Author

@simonpoole simonpoole commented Oct 31, 2019

@thomersch IMHO it is quite possible that the OSMF could pay the dev for help in/doing the migration and any missing bits that might need to be developed. Maybe you could start a conservation with them/him about this?

@thomersch
Copy link
Member

@thomersch thomersch commented Oct 31, 2019

@simonpoole Sure, will do

@pnorman
Copy link
Collaborator

@pnorman pnorman commented Feb 23, 2020

@simonpoole Sure, will do

@thomersch did the conversation with the askbot developer happen?

@PeterMortensen
Copy link

@PeterMortensen PeterMortensen commented Mar 23, 2020

There is the upcoming Codidact - codidact.org (open source splinter from Stack Exchange).

The meta site for it is on Discourse, but it is not based on Discourse.

@grischard
Copy link

@grischard grischard commented Apr 10, 2020

@mwoehlke-kitware
Copy link

@mwoehlke-kitware mwoehlke-kitware commented Jul 3, 2020

BTW, I was pointed over here from this question. I'd like to point out that the current site is just plain broken by third-party cookie blocking, which is a fairly minimal thing for anyone even remotely privacy-conscious to enable, and can't be easily disabled on a per-site basis. (This is because it relies on Google for jQuery, and Google won't serve that unless they can set their own cookies. Privacy-conscious folks should be screaming in horror about now...) The only thing I can do on the site is ask questions. I can't comment, can't vote, and suspect I can't even answer, and previews and tags are completely non-functional.

Despite comments here to the contrary, the current OSQA instance cannot be considered functional by reasonable definitions.

@Kovoschiz
Copy link

@Kovoschiz Kovoschiz commented Jul 4, 2020

BTW, I was pointed over here from this question. I'd like to point out that the current site is just plain broken by third-party cookie blocking, which is a fairly minimal thing for anyone even remotely privacy-conscious to enable, and can't be easily disabled on a per-site basis. (This is because it relies on Google for jQuery, and Google won't serve that unless they can set their own cookies. Privacy-conscious folks should be screaming in horror about now...) The only thing I can do on the site is ask questions. I can't comment, can't vote, and suspect I can't even answer, and previews and tags are completely non-functional.

Despite comments here to the contrary, the current OSQA instance cannot be considered functional by reasonable definitions.

Fyi, tomhughes/osqa@577d530 appears to fix the thing mostly, as commented in that question later. Answering, reporting, and editing (forgot about favoriting) worked fine before. See http://gis.19327.n8.nabble.com/our-Q-A-site-help-openstreetmap-org-is-dying-tp5966020p5969974.html as well.

@cellio
Copy link

@cellio cellio commented Dec 11, 2020

Hello. Codidact team member here. We might be able to help you if you're still interested in setting up your community elsewhere. We have a hosted network of communities and you're welcome to propose yours, or if you're looking for self-hosting, our project is open-source but nobody else has set it up yet, so we don't have a nice package with good documentation that you can just unpack and go. https://codidact.org/ has links for our Discord server (for chat), our GitHub repo, and our existing communities including our network-wide meta. Please feel free to get in touch if you want to explore options.

If this is already settled and no longer of interest, I apologize for the intrusion.

@GoodClover
Copy link

@GoodClover GoodClover commented Dec 11, 2020

Just mentioning that from the Codidact Discord there was link for these scripts as a basis for seeing how migrating data may be possible.

They are for importing Stack Exchange data, written in Ruby so I don't know much about how they work.

@ArtOfCode-
Copy link

@ArtOfCode- ArtOfCode- commented Dec 12, 2020

Just further from @cellio's comment - more than happy to provide assistance with installing a self-hosted instance of Codidact if that's what y'all would be looking for. If it's easier to have a community on our existing network, we can help with content import and custom domains, etc, as necessary.

@natrius
Copy link

@natrius natrius commented Mar 13, 2021

What are the features that are expected by you guys from a QA? Because some things can be done with discourse. I'm just interested in what is expected because it is also part of the discussion #377 there.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet