Display a warning message on old (possibly outdated) wiki pages #290
Conversation
…rch. Instead of looking for a specific haros .yaml file, now the test_results/ folder is searched for all kinds of test result reports.
When a wiki page was not updated recently, a warning message is displayed in the header, informing the reader that the information on the wiki page may be outdated. The default threshold for the warning to appear is 365 days, but can be adjusted by adding `outdated_page_warning_threshold = <days>` to the config file.
By mistake, unrelated commits ended up in this branch. This is a rollback to undo the changes so that the PR remains purely about the obsolete page warning.
|
Thanks @max-krichenbauer and @droter for the effort. I think @gavanderhoorn recommended a text bot the warning message. See the text below: "This page was last edited on YYYY-MM-DD, which is XX days ago. Please be aware that it may not reflect the current state of development or state-of-the-art any longer and commands or screenshots shown could no longer work or be outdated. Help the community by editing this page and correcting or improving things where necessary. |
|
I would question the general assumption that the last edit date has any reasonable correlation of the page content being outdated. I am pretty sure there are numerous pages which haven't been touched for years but their content is still as valid / relevant / up-to-date as it needs to be. And also some pages might have recently been touched for minor changes but still contain outdated content. Therefore I don't think this generic note based on the last edit timestamp is something which should be done. |
|
Some context: I've been thinking about this for quite some time now, and as we were looking for small tasks with potential high ROI in the SQWG, I suggested adding a notice (actually a static This question on ROS Answers made me finally suggest this for the SQWG: How to use protobuf with ros?. The OP there appeared to have found a page on the wiki which gave the impression that "Google Protocol Buffers" was something that could be used with ROS. Even though the page was in the
"hours" is probably a slight exaggeration on his/her part, but only finding out after quite some time that something is no longer applicable is not a good thing. As part of answering his question, I decided to add an admonition to the page (and its parent) to clarify the state of the page and the information on it.
As it would appear there aren't sufficient editors for the ROS wiki to make sure information stays up to date or pages are kept relevant, the idea is that at least some indication of the state of a page would already be much better than no indication at all. While I agree that adding a blanket statement of "outdatedness" based on last time something was edited may not be accurate in all cases, it would seem to allow us to show some indication to users that they would need to keep in mind that information may not be completely up-to-date any longer.
@dirk-thomas: if you have ideas for a better threshold or criteria that would be great. It would also be great if we could add something like this to the wiki, as one of the bigger frustrations we've heard about from newcomers to the community is that it's completely unclear whether something (on the wiki, in repositories or on other pages) is still relevant, to which ROS versions it applies, where to find information on what still applies, how to judge whether something is even relevant at all, etc, etc. And this frustration can lead to users turning away completely. Note that the original intent was not to mark pages as unconditionally outdated or irrelevant, but to post a friendly notice that stated something like:
Again: not showing anything is essentially saying: this page documents the state-of-the-art and everything on it should work. (No, of course it is not exactly the same, but the wiki is considered by many newcomers to be "the official documentation of ROS") We should try to avoid late user-facing errors and the resulting frustration by making users aware of the potential of incomplete or partially outdated information. A simple notice like this could avoid users taking things at face value and waste time and effort. |
👍 adding that annotation to pages where the content is outdated is a good idea.
I don't think that there is any automation / threshold which can achieve that kind if flagging. False positives (marking pages as outdated if they are not) is imo as confusing as not having a note on outdated pages. Imo this needs to be a manual process to determine if the content is stale. |
While I agree in principle, I believe it's just not feasible. That leaves us with either doing nothing, or adding a little automation that shows it everywhere it makes sense. Age is certainly not the only nor a perfect indication something is not up-to-date, but it is a good first one. That's what the notice is about: "user, note this page is such-and-so days old. This may mean there could be things that are not exactly as you read them here". Note again: I'm not saying we should add a red warning stating unconditionally the page is outdated. The actual text the notice displays very much influences how such a notice would be understood. |
|
Thank you all for your comments! I updated the text as suggested. Regarding whether the last-edit-date is a suitable heuristic on whether the content is outdated, I believe the key lies in the threshold. If an article is several years old, it's getting really likely that the project was abandoned and/or relates to EOL technologies (old versions of OS, distros or programming languages that are no longer supported). From my perspective of just recently finding my way into ROS I can relate to @gavanderhoorn 's examples. I too was (and often still am) confused by articles and tutorials that just don't work anymore and sometimes break your system when trying to use them. It could also serve as a motivation to authors the revisit their article at least once a year to check if anything has changed. [EDIT:] We could facilitate the last point I mentioned by changing "Help the community by editing this page" (most people reading the article precisely do so because they are lacking knowledge, so they might not be able to edit it anyway) to "Please contact the author to ensure that the information is still up to date". Please let me know what you think. |
|
@max-krichenbauer: thanks for updating the message. Could you perhaps also update the screenshot you show in the OP? |
|
@gavanderhoorn Sure, but which message should I put in there now? |
|
Whatever the message is that the code in this PR at this point now shows. |
|
And to get some more opinions on this we might want to post to the Quality category on Discourse. There could be readers there with other or better ideas on how to detect staleness of content on the wiki. |
|
Here's the screenshot of the current version of this PR: |
There was a problem hiding this comment.
I don't particularly think that this is a good metric to try to automatically categorize content. There are many different types of pages.
Some pages were written once and forgotten. Other pages are much more mature have been revised many times and converged on a stable straight forward solution.
On the other end of the spectrum, if things are recently edited they may have been changed incorrectly by a new user. As an example I definitely have tutorials which have been stable for a long time and then someone comes along and tries to hepfully fix a problem that was caused by something like an incorrect setup of prerequisites. For example a new user who finds that everything just works if they add this random word sudo to all commands in the tutorial and edits the tutorial to be helpful for following users.
We already have the date of last editing displayed on every page. We could make it more promenant. But I don't think that there's any way that we can editorialize on that without having a significant false positive rate.
In particular what threshold would you
A quick survey of the top 10 pages by visits with their last edit date are:
- /ROS/Tutorials 2019-12-01
- / 2018-11-29
- /kinetic/Installation/Ubuntu 2019-07-25
- /melodic/Installation/Ubuntu 2019-07-25
- /ROS/Installation 2018-06-07
- /ROS/Tutorials/InstallingandConfiguringROSEnvironment 2017-05-22
- /melodic/Installation 2018-05-23
- /kinetic/Installation 2018-01-08
- /ROS/Tutorials/CreatingPackage 2013-05-30
- /ROS/Tutorials/WritingPublisherSubscriber(c++) 2019-07-18
Note of these pages are valid for that warning. And as you can see age does not mean that it's necessarily out of date. To make this work for just these cases you're setting the threshold at 5+ years. And it still doesn't differentiate between typo fixes vs real content fixes etc. I see this as basically incentivizing people to touch files a lot to keep the warning away. And that seem like making work not improving our process or making our product or experience better.
A agree that there is outdated content. But this specific example I don't have a lot of sympathy for the miss understanding. The first sentence was "This page summarizes various serialization libraries and their pros and cons." And it's a list with various comparisons. You can spent an unlimited amount of time trying to find things that don't exist, it's just a matter of how long you are willing to keep searching before you decide you've searched enough. I think that the fact that he only found the single reference that definitely did not state anything about it working was actually a pretty telling result.
|
@tfoote: something which may not be clear -- it wasn't for me, although I'd read the code: this notice is only shown for pages that contain the So it essentially comes down to package authors and/or maintainers not having kept the pages of their packages up-to-date. In a way, hiding pages behind the version rocker and having the "show EOL distributions" is something that could be compared to the notice proposed in this PR. |
For any threshold you choose there will be cases where it is simply wrong - either resulting in a false positive or a false negative. As mentioned on Discourse I would suggest to add a neutral "Age" information to the top of the page to inform reader when the content was last edited (probably with a link to the detailed history). |
|
Thank you all for your comments! Much appreciated. For the convenience of people finding this thread, here's the link: Here's a screenshot of the new message: |
In the discussion on Discourse, the worry of some maintainers that the warning message would cause too many emails to maintainers even when the content was not outdated was addressed by changing the wording to "if you run into problems ..."
|
Based on the discussion over at Discourse, I changed the wording again: If you run into problems, please contact the author or package maintainer to ensure that the content is still up-to-date. This should now achieve both of the conflicting demands: Please let me know what you think. |
|
@gavanderhoorn wrote:
... which means that this PR doesn't even apply to the Protobuf wiki page that lead to your suggesion of this PR. :) For reference: There were some comments on the Discourse thread that suggested an alternative to this PR: using badges to show that the contents of a wiki page were tested to work with a specific ROS version. I very much prefer that idea to the current proposal. |
This recommendation goes against the general ROS support guidelines: http://wiki.ros.org/Support#Do_Not |
|
@dirk-thomas Thank you for the comment. I wasn't aware of this. |
|
Since it seems the majority's opinion is that it's not a good idea to display this warning, I'm closing this PR. |
|
Thanks for all your effort on this @max-krichenbauer 👍 |
In this PR, when a wiki page was not updated recently, a warning message is displayed in the header, informing the reader that the information on the wiki page may be outdated.
The default threshold for the warning to appear is 365 days, but can be adjusted by adding
outdated_page_warning_threshold = <days>to the config file.
(in the screenshot, the threshold was set to zero days for testing)
This PR is part of a ROS Quality Assurance Workgroup initiative and based on a suggestion from @gavanderhoorn