fix Qubes documentation edit cumbersomeness #3629

adrelanos · Feb 25, 2018

I am spoiled by the relative ease of (media)wiki edits. One can press the edit button, the edit maybe gets confirmed by an admin, and then goes live. Edits can be done from the comfort of the browser window which has a functional spell checker as well as search function. Also one can edit one section (headline) only and doesn't have to navigate all the wiki source.

When you go to some and press edit, one edits with github's editor.

full page edit (hard to navigate) rather than section
broken browser spell checker in github editor
broken browser search function in github editor
lots of clicks and page loading times

Another thing which is more a policy rather than technical thing, I think:
[1] Why not merge edits with spelling mistakes? Even if a spell mistake goes online for a few minutes to hours, if it's not crucial or insecure, I would say just merge and add spelling fixes on top. Because after a git pull request adding a new commit on top is also a lot overhead.

I am also a daily user of git so I could make edits using git and a local editor. However, for me creating a bit branch for simple documentation changes is overkill as well. A lot overhead typing.

In result this leads to me getting discouraged, making much fewer edits than I would. The usual for me is to always improve anything that confused me earlier one for the benefit of the community as well as myself. Big win-win in the spirit of Libre Software.

Now, I am not the center of the world, however, I was wondering if others have a similar view. I also know users who are both Whonix and Qubes users, that contribute a lot to Whonix wiki, but not to Qubes documentation. It might be, that is the reason.

Suggestions:

Can this policy be eased please?
I was looking into prose.io but it also breaks the browser's spell checker. Are there any alternatives?

One can press the edit button, the edit maybe gets confirmed by an admin, and then goes live.

For security, all PRs have to be reviewed prior to being merged.

Edits can be done from the comfort of the browser window which has a functional spell checker as well as search function. [...] lots of clicks and page loading times

I agree that the (optional) GitHub editor interface could be improved and that the editing workflow could be easier for contributors. I welcome solutions that improve this part of the workflow.

Another thing which is more a policy rather than technical thing, I think:
[1] Why not merge edits with spelling mistakes?

We do when the choice is between merging good content that contains spelling mistakes vs. not merging it at all. This is consistent with first asking the contributor to fix the spelling mistakes, which they are almost always willing to do. We do this because we want to maintain high quality standards for the documentation.

Even if a spell mistake goes online for a few minutes to hours, if it's not crucial or insecure,

Sure, but that doesn't mean we shouldn't still try to fix them before the content goes live, when feasible.

I would say just merge and add spelling fixes on top. Because after a git pull request adding a new commit on top is also a lot overhead.

The contributor submitting a new PR to fix the spelling mistakes from a previously merged PR would be even more overhead than the contributor just adding a commit to the existing PR before we merge it. The onus is on the contributor to fix their own spelling mistakes, because adopting the policy that we'll fix any mistakes encourages the submission of content that has not been proofread by the author. (This already happens too often.)

I am also a daily user of git so I could make edits using git and a local editor. However, for me creating a bit branch for simple documentation changes is overkill as well. A lot overhead typing.

Perhaps you could consider creating some aliases to reduce the required amount of typing. I personally find this approach helpful.

In result this leads to me getting discouraged, making much fewer edits than I would. The usual for me is to always improve anything that confused me earlier one for the benefit of the community as well as myself. Big win-win in the spirit of Libre Software.

Now, I am not the center of the world, however, I was wondering if others have a similar view. I also know users who are both Whonix and Qubes users, that contribute a lot to Whonix wiki, but not to Qubes documentation. It might be, that is the reason.

We don't want to discourage contributions, but we also don't want to sacrifice the security of the documentation system. This seems like a classic trade-off between security and convenience. If you have any ideas for easing the workflow without sacrificing security, please let us know.

Can this policy be eased please?

Which policy?

Security (e.g., all PRs must be reviewed before being merged): very unlikely to change
Merging PRs with typos: the actual policy does not seem to contradict your stated desire (see above)
Other: please clarify

I was looking into prose.io but it also breaks the browser's spell checker. Are there any alternatives?

I would also like to know.

andrewdavidwong · Feb 25, 2018

One can press the edit button, the edit maybe gets confirmed by an admin, and then goes live.

For security, all PRs have to be reviewed prior to being merged.

Edits can be done from the comfort of the browser window which has a functional spell checker as well as search function. [...] lots of clicks and page loading times

I agree that the (optional) GitHub editor interface could be improved and that the editing workflow could be easier for contributors. I welcome solutions that improve this part of the workflow.

Another thing which is more a policy rather than technical thing, I think:
[1] Why not merge edits with spelling mistakes?

We do when the choice is between merging good content that contains spelling mistakes vs. not merging it at all. This is consistent with first asking the contributor to fix the spelling mistakes, which they are almost always willing to do. We do this because we want to maintain high quality standards for the documentation.

Even if a spell mistake goes online for a few minutes to hours, if it's not crucial or insecure,

Sure, but that doesn't mean we shouldn't still try to fix them before the content goes live, when feasible.

I would say just merge and add spelling fixes on top. Because after a git pull request adding a new commit on top is also a lot overhead.

The contributor submitting a new PR to fix the spelling mistakes from a previously merged PR would be even more overhead than the contributor just adding a commit to the existing PR before we merge it. The onus is on the contributor to fix their own spelling mistakes, because adopting the policy that we'll fix any mistakes encourages the submission of content that has not been proofread by the author. (This already happens too often.)

I am also a daily user of git so I could make edits using git and a local editor. However, for me creating a bit branch for simple documentation changes is overkill as well. A lot overhead typing.

Perhaps you could consider creating some aliases to reduce the required amount of typing. I personally find this approach helpful.

In result this leads to me getting discouraged, making much fewer edits than I would. The usual for me is to always improve anything that confused me earlier one for the benefit of the community as well as myself. Big win-win in the spirit of Libre Software.

Now, I am not the center of the world, however, I was wondering if others have a similar view. I also know users who are both Whonix and Qubes users, that contribute a lot to Whonix wiki, but not to Qubes documentation. It might be, that is the reason.

We don't want to discourage contributions, but we also don't want to sacrifice the security of the documentation system. This seems like a classic trade-off between security and convenience. If you have any ideas for easing the workflow without sacrificing security, please let us know.

Can this policy be eased please?

Which policy?

Security (e.g., all PRs must be reviewed before being merged): very unlikely to change
Merging PRs with typos: the actual policy does not seem to contradict your stated desire (see above)
Other: please clarify

I was looking into prose.io but it also breaks the browser's spell checker. Are there any alternatives?

I would also like to know.

The thing I find most annoying about the process is Github editor's lack of text keyboard selection. Could be my choice of browser and security settings too, haven't been that annoyed yet to research further...

awokd · Feb 26, 2018

The thing I find most annoying about the process is Github editor's lack of text keyboard selection. Could be my choice of browser and security settings too, haven't been that annoyed yet to research further...

I was looking into prose.io but it also breaks the browser's spell checker. Are there any alternatives?

I've had a quick play today with https://forestry.io - when editing content it does not appear to break Firefox built in spell check. Not overly keen on the level of access it seems to want, but in my case it wouldn't be a problem as I only use this account for tickets and documentation submissions...

Edit - looks like they use something called prosemirror https://discuss.prosemirror.net/t/the-forestry-io-markdown-editor/1164

vincentadultman · Feb 26, 2018

I was looking into prose.io but it also breaks the browser's spell checker. Are there any alternatives?

I've had a quick play today with https://forestry.io - when editing content it does not appear to break Firefox built in spell check. Not overly keen on the level of access it seems to want, but in my case it wouldn't be a problem as I only use this account for tickets and documentation submissions...

Edit - looks like they use something called prosemirror https://discuss.prosemirror.net/t/the-forestry-io-markdown-editor/1164

FWIW my experience after sending my first doc PR a few days ago is that the whole process isn't too obvious for git noobs like me. For instance:

I've forked the qubes-doc repo a long time ago but couldn't figure how to "resync" it with qubes'. A reply to a similar question on stack overflow got ~3000 thumbs up and half a million view so it looks like it's something people expect to do but doesn't fit in git's workflow. I ended up deleting the whole repo and re-forking it.
It is recommended to strip the leading http://qubes-os.org in links to make them relative for off-line usage; but then how to test those links when the doc sits in a forked repo ? (to be honest, I didn't try to find how).
There were some travis CI errors. I had no idea what that stuff was (I do now).
My PR was bigger than expected and I wanted to send several incremental PRs based on my commits (the idea was to submit PR 1/4, PR 2/4, ..., like bigger patches in other open source projects). With diff/patch I'd do that in 30 seconds. I've spent half an hour trying to do the same with git and eventually gave out. That's probably something that is very easy to do but it requires some understanding of git.

tl;dr;, having git/github knowledge as a prerequisite likely discourages first-time contributors.

In a ML post I suggested creating a 'staging' wiki to work around that issue. At first glance it looked like a good solution but there are many ways this could go wrong (lack of synchronization with the official docs, crappy/unverified content, no PRs pushed to the official doc anymore, etcetera). I've reverted to using issues to discuss about documentation before submitting a PR, this has worked well but this assumes people look at issues, which is likely not the case.

So - I'm complaining here but at the same time I don't really see how this could be improved. Maybe a wiki that is a mirror of the official docs would help: people could use it to edit (with acks from doc admins) and the admins would then submit PRs to the official doc. Alternatively, emails could be sent to a qubesdoc@ adress and a team of volunteers/admins could transform them into PRs. But for either way to work there must be enough volunteers willing to spend time.

taradiddles · Feb 26, 2018

FWIW my experience after sending my first doc PR a few days ago is that the whole process isn't too obvious for git noobs like me. For instance:

I've forked the qubes-doc repo a long time ago but couldn't figure how to "resync" it with qubes'. A reply to a similar question on stack overflow got ~3000 thumbs up and half a million view so it looks like it's something people expect to do but doesn't fit in git's workflow. I ended up deleting the whole repo and re-forking it.
It is recommended to strip the leading http://qubes-os.org in links to make them relative for off-line usage; but then how to test those links when the doc sits in a forked repo ? (to be honest, I didn't try to find how).
There were some travis CI errors. I had no idea what that stuff was (I do now).
My PR was bigger than expected and I wanted to send several incremental PRs based on my commits (the idea was to submit PR 1/4, PR 2/4, ..., like bigger patches in other open source projects). With diff/patch I'd do that in 30 seconds. I've spent half an hour trying to do the same with git and eventually gave out. That's probably something that is very easy to do but it requires some understanding of git.

tl;dr;, having git/github knowledge as a prerequisite likely discourages first-time contributors.

In a ML post I suggested creating a 'staging' wiki to work around that issue. At first glance it looked like a good solution but there are many ways this could go wrong (lack of synchronization with the official docs, crappy/unverified content, no PRs pushed to the official doc anymore, etcetera). I've reverted to using issues to discuss about documentation before submitting a PR, this has worked well but this assumes people look at issues, which is likely not the case.

So - I'm complaining here but at the same time I don't really see how this could be improved. Maybe a wiki that is a mirror of the official docs would help: people could use it to edit (with acks from doc admins) and the admins would then submit PRs to the official doc. Alternatively, emails could be sent to a qubesdoc@ adress and a team of volunteers/admins could transform them into PRs. But for either way to work there must be enough volunteers willing to spend time.

It is recommended to strip the leading http://qubes-os.org in links to make them relative for off-line usage; but then how to test those links when the doc sits in a forked repo ? (to be honest, I didn't try to find how).

You should be able to test them by hosting the site locally:

https://github.com/QubesOS/qubesos.github.io#instructions

tl;dr;, having git/github knowledge as a prerequisite likely discourages first-time contributors.

I think it would be very helpful to have the knowledge you've gained available in the the doc contribution instructions. It would probably ease the learning curve for others. Would you be willing to submit a PR?

andrewdavidwong · Feb 26, 2018

It is recommended to strip the leading http://qubes-os.org in links to make them relative for off-line usage; but then how to test those links when the doc sits in a forked repo ? (to be honest, I didn't try to find how).

You should be able to test them by hosting the site locally:

https://github.com/QubesOS/qubesos.github.io#instructions

tl;dr;, having git/github knowledge as a prerequisite likely discourages first-time contributors.

I think it would be very helpful to have the knowledge you've gained available in the the doc contribution instructions. It would probably ease the learning curve for others. Would you be willing to submit a PR?

> Can this policy be eased please? Which policy?

Merging with typos.

> I would say just merge and add spelling fixes on top. Because after a git pull request adding a new commit on top is also a lot overhead. The contributor submitting a new PR to fix the spelling mistakes from a previously merged PR would be even more overhead than the contributor just adding a commit to the existing PR before we merge it.

Adding a commit is hard. Usually I do it like this: 1) I'll go back to QubesOS/qubes-doc#582 2) click on files changed 3) click on view 4) end up on https://github.com/adrelanos/qubes-doc/blob/f8c0eac48df36c1bbd25235481a36ed000d83aed/security/yubi-key.md 5) figure out I have to go to the branch 6) open another tab going to QubesOS/qubes-doc#582 7) Figure out the branch name: patch-47 8) go back to original browser tab and choose patch-47 from long list 9) finally make the edit ...or do all of it using git.

The onus is on the contributor to fix their own spelling mistakes, because adopting the policy that we'll fix any mistakes encourages the submission of content that has not been proofread by the author. (This already happens too often.)

I see documentation like a layered approach. It starts somewhere imperfect and with every revision on top it gets better. To keep individual diff's look easier, as well as to get non-controversial stuff in so at least that is done, I also like splitting into multiple smaller changes. Like for yubikey I had more improvements in mind back then in November 2017. But now some time has passed and my knowledge on the subject has faded a bit since so now it's more effort to reinvent it. A delayed merge can also lead to merge conflict (like in yubykey case). Other edits are blocked until the author of the original pull request is done. And resolving a merge conflict is even more cumbersome. Also other imperfections if feasible. Rather than posting the suggestion as a comment, I would suggest to just merge "to staging". (Meaning, merging without that version going live.) Apply the suggestion and go live. And even if it goes live for a moment with typos, it's may be still better than blocking the replacement of the old version. Like in the case of yubikey: old version, outdated, more complicated: https://github.com/QubesOS/qubes-doc/blob/58ef892234098b2228c8ad6ab88ccf0e754883c7/security/yubi-key.md a lot better, I hope, but a few typos: https://github.com/QubesOS/qubes-doc/blob/f8c0eac48df36c1bbd25235481a36ed000d83aed/security/yubi-key.md

> In result this leads to me getting discouraged, making much fewer edits than I would. The usual for me is to always improve anything that confused me earlier one for the benefit of the community as well as myself. Big win-win in the spirit of Libre Software. > > Now, I am not the center of the world, however, I was wondering if others have a similar view. I also know users who are both Whonix and Qubes users, that contribute a lot to Whonix wiki, but not to Qubes documentation. It might be, that is the reason. We don't want to discourage contributions, but we also don't want to sacrifice the security of the documentation system. This seems like a classic trade-off between security and convenience. If you have any ideas for easing the workflow without sacrificing security, please let us know.

Of course keep review before it goes live and block anything. Typos and other imperfections aren't security and in these cases I would appreciate convenience.

adrelanos · Feb 26, 2018

> Can this policy be eased please? Which policy?

Merging with typos.

> I would say just merge and add spelling fixes on top. Because after a git pull request adding a new commit on top is also a lot overhead. The contributor submitting a new PR to fix the spelling mistakes from a previously merged PR would be even more overhead than the contributor just adding a commit to the existing PR before we merge it.

Adding a commit is hard. Usually I do it like this: 1) I'll go back to QubesOS/qubes-doc#582 2) click on files changed 3) click on view 4) end up on https://github.com/adrelanos/qubes-doc/blob/f8c0eac48df36c1bbd25235481a36ed000d83aed/security/yubi-key.md 5) figure out I have to go to the branch 6) open another tab going to QubesOS/qubes-doc#582 7) Figure out the branch name: patch-47 8) go back to original browser tab and choose patch-47 from long list 9) finally make the edit ...or do all of it using git.

The onus is on the contributor to fix their own spelling mistakes, because adopting the policy that we'll fix any mistakes encourages the submission of content that has not been proofread by the author. (This already happens too often.)

I see documentation like a layered approach. It starts somewhere imperfect and with every revision on top it gets better. To keep individual diff's look easier, as well as to get non-controversial stuff in so at least that is done, I also like splitting into multiple smaller changes. Like for yubikey I had more improvements in mind back then in November 2017. But now some time has passed and my knowledge on the subject has faded a bit since so now it's more effort to reinvent it. A delayed merge can also lead to merge conflict (like in yubykey case). Other edits are blocked until the author of the original pull request is done. And resolving a merge conflict is even more cumbersome. Also other imperfections if feasible. Rather than posting the suggestion as a comment, I would suggest to just merge "to staging". (Meaning, merging without that version going live.) Apply the suggestion and go live. And even if it goes live for a moment with typos, it's may be still better than blocking the replacement of the old version. Like in the case of yubikey: old version, outdated, more complicated: https://github.com/QubesOS/qubes-doc/blob/58ef892234098b2228c8ad6ab88ccf0e754883c7/security/yubi-key.md a lot better, I hope, but a few typos: https://github.com/QubesOS/qubes-doc/blob/f8c0eac48df36c1bbd25235481a36ed000d83aed/security/yubi-key.md

> In result this leads to me getting discouraged, making much fewer edits than I would. The usual for me is to always improve anything that confused me earlier one for the benefit of the community as well as myself. Big win-win in the spirit of Libre Software. > > Now, I am not the center of the world, however, I was wondering if others have a similar view. I also know users who are both Whonix and Qubes users, that contribute a lot to Whonix wiki, but not to Qubes documentation. It might be, that is the reason. We don't want to discourage contributions, but we also don't want to sacrifice the security of the documentation system. This seems like a classic trade-off between security and convenience. If you have any ideas for easing the workflow without sacrificing security, please let us know.

Of course keep review before it goes live and block anything. Typos and other imperfections aren't security and in these cases I would appreciate convenience.

Adding a commit is hard. Usually I do it like this:

I'll go back to QubesOS/qubes-doc#582

click on files changed

At this point you can click edit icon instead of "view"...

marmarek · Feb 26, 2018

Adding a commit is hard. Usually I do it like this:

I'll go back to QubesOS/qubes-doc#582

click on files changed

At this point you can click edit icon instead of "view"...

@andrewdavidwong

I think it would be very helpful to have the knowledge you've gained available in the the doc contribution instructions. It would probably ease the learning curve for others. Would you be willing to submit a PR?

I'd be happy to but I really don't feel I've gained enough knowledge; maybe once I have a bit more experience.

taradiddles · Feb 27, 2018

@andrewdavidwong

I think it would be very helpful to have the knowledge you've gained available in the the doc contribution instructions. It would probably ease the learning curve for others. Would you be willing to submit a PR?

I'd be happy to but I really don't feel I've gained enough knowledge; maybe once I have a bit more experience.

...or do all of it using git.

Isn't it easier with git, since you can just keep pushing to the same branch?

I see documentation like a layered approach. It starts somewhere
imperfect and with every revision on top it gets better.

To keep individual diff's look easier, as well as to get
non-controversial stuff in so at least that is done, I also like
splitting into multiple smaller changes.

👍

Like for yubikey I had more improvements in mind back then in November
2017. But now some time has passed and my knowledge on the subject has
faded a bit since so now it's more effort to reinvent it.

But you could have just included those in the original PR, separate PRs, or your own personal set of notes, right?

A delayed merge can also lead to merge conflict (like in yubykey case).
Other edits are blocked until the author of the original pull request is
done.

Maybe I'm misunderstanding you, but I don't think that's how it works. The fact that a PR has a merge conflict in a particular file does not block other edits to that file (only merging that PR itself).

And resolving a merge conflict is even more cumbersome.

Yes, but isn't this a problem with any system (even pencil and paper) in which multiple authors are allowed to edit their own copy of a document before merging them? (How do wikis handle this? Google Docs seems to handle it by showing all changes to a single copy in real time and not allowing separate copies to be mergeed together.)

Also other imperfections if feasible. Rather than posting the suggestion
as a comment, I would suggest to just merge "to staging". (Meaning,
merging without that version going live.) Apply the suggestion and go live.

I can see the appeal. If I understand correctly, one of the main goals of such a system is to reduce merge conflicts:

Current system:

PR-1 submitted against foo.md.
PR-2 submitted against foo.md (conflicts with PR-1).
PR-2 merged.
PR-1 now has a merge conflict.

Staging system:

PR-1 submitted against foo.md.
PR-1 merged to staging.
PR-2 submitted against foo.md (based on foo.md in staging, so no conflicts).
PR-2 merged to staging.
PR-1 merged to master.
PR-2 merged to master.

Is this the idea?

Of course, the problem with the current system could still reappear at the staging level (i.e., two conflicting PRs are submitted before either one gets merged to staging), but I guess the idea is that merges to staging will happen faster than merges to live currently do. That's probably true, but I don't know whether it'll be fast enough to make a noticeable difference now that we're reviewing PRs in a more timely fashion than we were a few months ago.

@marmarek, what do you think?

And even if it goes live for a moment with typos, it's may be still
better than blocking the replacement of the old version.

But now we're no longer talking about the "staging" system, because stuff in staging isn't live, right?

Typos and other imperfections aren't security and in these cases I would
appreciate convenience.

Right, they aren't security issues, but I'm afraid that something like this will happen:

Contributor submits PR with good content but many typos.
We merge and say, "Please submit another PR that fixes the typos."
Contributor delays or never does so. The misspellings are effectively whitelisted by Travis CI (until fixed, if ever), allowing more misspellings to go through in the future.
Steps 1-3 keep repeating.
Eventually, the overall quality of the documentation is significantly lower.

This could be mitigated by doing a periodic sweep of all recently changed files (or just all of the documentation) with a spellchecker, but that requires the existence of people (heroes, really) with the time and inclination to do this. History has shown that such exemplary individuals do exist, though, so there's hope. 🙂

andrewdavidwong · Feb 28, 2018

...or do all of it using git.

Isn't it easier with git, since you can just keep pushing to the same branch?

I see documentation like a layered approach. It starts somewhere
imperfect and with every revision on top it gets better.

To keep individual diff's look easier, as well as to get
non-controversial stuff in so at least that is done, I also like
splitting into multiple smaller changes.

👍

Like for yubikey I had more improvements in mind back then in November
2017. But now some time has passed and my knowledge on the subject has
faded a bit since so now it's more effort to reinvent it.

But you could have just included those in the original PR, separate PRs, or your own personal set of notes, right?

A delayed merge can also lead to merge conflict (like in yubykey case).
Other edits are blocked until the author of the original pull request is
done.

Maybe I'm misunderstanding you, but I don't think that's how it works. The fact that a PR has a merge conflict in a particular file does not block other edits to that file (only merging that PR itself).

And resolving a merge conflict is even more cumbersome.

Yes, but isn't this a problem with any system (even pencil and paper) in which multiple authors are allowed to edit their own copy of a document before merging them? (How do wikis handle this? Google Docs seems to handle it by showing all changes to a single copy in real time and not allowing separate copies to be mergeed together.)

Also other imperfections if feasible. Rather than posting the suggestion
as a comment, I would suggest to just merge "to staging". (Meaning,
merging without that version going live.) Apply the suggestion and go live.

I can see the appeal. If I understand correctly, one of the main goals of such a system is to reduce merge conflicts:

Current system:

PR-1 submitted against foo.md.
PR-2 submitted against foo.md (conflicts with PR-1).
PR-2 merged.
PR-1 now has a merge conflict.

Staging system:

PR-1 submitted against foo.md.
PR-1 merged to staging.
PR-2 submitted against foo.md (based on foo.md in staging, so no conflicts).
PR-2 merged to staging.
PR-1 merged to master.
PR-2 merged to master.

Is this the idea?

Of course, the problem with the current system could still reappear at the staging level (i.e., two conflicting PRs are submitted before either one gets merged to staging), but I guess the idea is that merges to staging will happen faster than merges to live currently do. That's probably true, but I don't know whether it'll be fast enough to make a noticeable difference now that we're reviewing PRs in a more timely fashion than we were a few months ago.

@marmarek, what do you think?

And even if it goes live for a moment with typos, it's may be still
better than blocking the replacement of the old version.

But now we're no longer talking about the "staging" system, because stuff in staging isn't live, right?

Typos and other imperfections aren't security and in these cases I would
appreciate convenience.

Right, they aren't security issues, but I'm afraid that something like this will happen:

Contributor submits PR with good content but many typos.
We merge and say, "Please submit another PR that fixes the typos."
Contributor delays or never does so. The misspellings are effectively whitelisted by Travis CI (until fixed, if ever), allowing more misspellings to go through in the future.
Steps 1-3 keep repeating.
Eventually, the overall quality of the documentation is significantly lower.

This could be mitigated by doing a periodic sweep of all recently changed files (or just all of the documentation) with a spellchecker, but that requires the existence of people (heroes, really) with the time and inclination to do this. History has shown that such exemplary individuals do exist, though, so there's hope. 🙂

(Closed by accident.)

andrewdavidwong · Feb 28, 2018

(Closed by accident.)

@andrewdavidwong Having looked at a travis run for a doc I see an example what you mean, so the tool in use is hunspell and new terms get auto ignored in future if the commit is approved. With regards the heroic act to which you refer, is there a whitelist file / personal list (I'm most used to aspell) of actual "Qubes Terms" to use in addition to the built in spell check dictionary or just one somewhere built in the imperfect manner you describe?

I've read https://blog.eleven-labs.com/en/how-to-check-the-spelling-of-your-docs-from-travis-ci/ but keen to see how it works for Qubes already with hunspell before I do anything to get an idea of the scale of the problem.

Edit - sorry I see the spelling issue is mostly hypothetical as care is currently taken, I believe the errors I've seen in the past could perhaps be more referred to as grammar errors

Also how does this interact with the translation project? Would you prefer the English docs as is use US English or British English?

vincentadultman · Mar 14, 2018

@andrewdavidwong Having looked at a travis run for a doc I see an example what you mean, so the tool in use is hunspell and new terms get auto ignored in future if the commit is approved. With regards the heroic act to which you refer, is there a whitelist file / personal list (I'm most used to aspell) of actual "Qubes Terms" to use in addition to the built in spell check dictionary or just one somewhere built in the imperfect manner you describe?

I've read https://blog.eleven-labs.com/en/how-to-check-the-spelling-of-your-docs-from-travis-ci/ but keen to see how it works for Qubes already with hunspell before I do anything to get an idea of the scale of the problem.

Edit - sorry I see the spelling issue is mostly hypothetical as care is currently taken, I believe the errors I've seen in the past could perhaps be more referred to as grammar errors

Also how does this interact with the translation project? Would you prefer the English docs as is use US English or British English?

Also how does this interact with the translation project?

There is currently no interaction. I expect that this is one of the issues the localization team will have to consider soon.

Would you prefer the English docs as is use US English or British English?

Either way is fine. For the sake of consistency, it might be nice to stick with one (US English is currently more common in our documentation), but I'm not convinced that this consideration is strong enough that we should ask any British English writer to change her ways.

andrewdavidwong · Mar 15, 2018

Also how does this interact with the translation project?

There is currently no interaction. I expect that this is one of the issues the localization team will have to consider soon.

Would you prefer the English docs as is use US English or British English?

Either way is fine. For the sake of consistency, it might be nice to stick with one (US English is currently more common in our documentation), but I'm not convinced that this consideration is strong enough that we should ask any British English writer to change her ways.

This problem may be mitigated by the Qubes Community Collaboration effort.

CC @Aekez, @awokd, @taradiddles

andrewdavidwong · Mar 18, 2018

This problem may be mitigated by the Qubes Community Collaboration effort.

CC @Aekez, @awokd, @taradiddles

This problem may be mitigated [...]

should :)

You kind of beat me to it - I was planning to post here at some point. We're making good progress but we're still ironing out a few things; hopefully we should be ready in a few days. But of course, any help/contributions/suggestions are welcome !

taradiddles · Mar 19, 2018

This problem may be mitigated [...]

should :)

You kind of beat me to it - I was planning to post here at some point. We're making good progress but we're still ironing out a few things; hopefully we should be ready in a few days. But of course, any help/contributions/suggestions are welcome !

Andrew David Wong:

> ...or do all of it using git. Isn't it easier with git, since you can just keep pushing to the same branch?

No. I find git cumbersome for something as easy as a wiki edit.

> I see documentation like a layered approach. It starts somewhere imperfect and with every revision on top it gets better. > > To keep individual diff's look easier, as well as to get > non-controversial stuff in so at least that is done, I also like > splitting into multiple smaller changes. 👍 > Like for yubikey I had more improvements in mind back then in November > 2017. But now some time has passed and my knowledge on the subject has > faded a bit since so now it's more effort to reinvent it. But you could have just included those in the original PR, separate PRs, or your own personal set of notes, right?

Original PR: No, since changes would be harder to review, since too many. Separate PR: Possible in theory but it would have broken. Given the likelihood of breaking, not very attractive. personal set of notes: Couldn't be expected that it would take so long and then there be merge conflicts.

> A delayed merge can also lead to merge conflict (like in yubykey case). > Other edits are blocked until the author of the original pull request is > done. Maybe I'm misunderstanding you, but I don't think that's how it works. The fact that a PR has a merge conflict in a particular file does not block other edits to that file (only merging that PR itself). > And resolving a merge conflict is even more cumbersome. Yes, but isn't this a problem with *any* system (even pencil and paper) in which multiple authors are allowed to edit their own copy of a document before merging them? (How do wikis handle this? Google Docs seems to handle it by showing all changes to a single copy in real time and not allowing separate copies to be mergeed together.)

With mediawiki, only small parts get changed. So as long as only one chapter is modified, there is no merge conflict.

> Also other imperfections if feasible. Rather than posting the suggestion > as a comment, I would suggest to just merge "to staging". (Meaning, > merging without that version going live.) Apply the suggestion and go live. I can see the appeal. If I understand correctly, one of the main goals of such a system is to reduce merge conflicts: Current system: 1. PR-1 submitted against `foo.md`. 2. PR-2 submitted against `foo.md` (conflicts with PR-1). 3. PR-2 merged. 4. PR-1 now has a merge conflict. Staging system: 1. PR-1 submitted against `foo.md`. 2. PR-1 merged to staging. 3. PR-2 submitted against `foo.md` (based on `foo.md` in staging, so no conflicts). 4. PR-2 merged to staging. 5. PR-1 merged to master. 6. PR-2 merged to master. Is this the idea?

Almost. Except: staging is merged into master when ready. Staging system: 1. PR-1 submitted against `foo.md`. 2. PR-1 merged to staging. 3. PR-2 submitted against `foo.md` (based on `foo.md` in staging, so no conflicts). 4. PR-2 merged to staging. 5. staging merged to master.

Of course, the problem with the current system could still reappear at the staging level (i.e., two conflicting PRs are submitted before either one gets merged to staging), but I guess the idea is that merges to staging will happen faster than merges to live currently do. That's probably true, but I don't know whether it'll be fast enough to make a noticeable difference now that we're reviewing PRs in a more timely fashion than we were a few months ago.

Yes.

@marmarek, what do you think?

Yes? :)

> And even if it goes live for a moment with typos, it's may be still > better than blocking the replacement of the old version. But now we're no longer talking about the "staging" system, because stuff in staging isn't live, right?

Live for a moment with typos or staging system is both much better.

> Typos and other imperfections aren't security and in these cases I would appreciate convenience. Right, they aren't security issues, but I'm afraid that something like this will happen: 1. Contributor submits PR with good content but many typos. 2. We merge and say, "Please submit another PR that fixes the typos." 3. Contributor delays or never does so. The misspellings are effectively whitelisted by Travis CI (until fixed, if ever), allowing more misspellings to go through in the future.

Partial solution: no auto whitelist?

4. Steps 1-3 keep repeating. 5. Eventually, the overall quality of the documentation is significantly lower.

I doubt that since there are other contributors like people "only" out to catch typos. (I guess people who wish to contribute but don't know what/how and therefore work on these things.)

adrelanos · Apr 14, 2018