Offline documentation

[rootkovska]

Make it possible to access the content of the documentation that is exposed on qubes-os.org (in practice: qubes-doc repo) on a new Qubes installation without requiring networking access.

One thing to consider: do we want to place the content of qubes-doc in Dom0 or in an (dedicated) AppVM?

[nrgaway]

I can create a VM with the Qubes management tool. This would allow the documentation to be updatable should network access become available.

Maybe this dedicated AppVM could also be used for other purposes as well?

[nrgaway]

I can create a VM with the Qubes management tool. This would allow the documentation to be updatable should network access become available.

Maybe this dedicated AppVM could also be used for other purposes as well?

[andrewdavidwong]

One thing to consider: do we want to place the content of qubes-doc in Dom0 or in an (dedicated) AppVM?

I think using a dedicated AppVM would be more secure. If malicious content somehow gets into qubes-doc, and the user attempts to read that content in dom0, this could result in dom0 being compromised, even if the user uses a "simple" command like less (see #1014). Note that we also have some untrusted image files in there.

In addition, having the content in an AppVM means that the user has the option of using various programs to render the Markdown source however she likes (including using Jekyll to create a local copy of the website) without having to install these extra programs in dom0, which would result in unnecessary bloat and increased attack surface.

[andrewdavidwong]

One thing to consider: do we want to place the content of qubes-doc in Dom0 or in an (dedicated) AppVM?

I think using a dedicated AppVM would be more secure. If malicious content somehow gets into qubes-doc, and the user attempts to read that content in dom0, this could result in dom0 being compromised, even if the user uses a "simple" command like less (see #1014). Note that we also have some untrusted image files in there.

In addition, having the content in an AppVM means that the user has the option of using various programs to render the Markdown source however she likes (including using Jekyll to create a local copy of the website) without having to install these extra programs in dom0, which would result in unnecessary bloat and increased attack surface.

[mfc]

I agree it should be part of a dedicated AppVM. Any updates on this @nrgaway ?

I think having a more general "offline-files" or "offline-docs" AppVM pre-populated with Qubes documentation would be awesome. Default displayed apps would probably be Help (going to qubes-doc content), Files, Terminal.

Users can decide if they want to use the VM to store other documents/files or just use it to read Qubes documentation when they are offline.

[mfc]

I agree it should be part of a dedicated AppVM. Any updates on this @nrgaway ?

I think having a more general "offline-files" or "offline-docs" AppVM pre-populated with Qubes documentation would be awesome. Default displayed apps would probably be Help (going to qubes-doc content), Files, Terminal.

Users can decide if they want to use the VM to store other documents/files or just use it to read Qubes documentation when they are offline.

[marmarek]

On Fri, Sep 04, 2015 at 04:47:39PM -0700, Michael Carbone wrote:

I agree it should be part of a dedicated AppVM. Any updates on this @nrgaway ?

I think having a more general "offline-files" or "offline-docs" AppVM pre-populated with Qubes documentation would be awesome. Default displayed apps would probably be Help (going to qubes-doc content), Files, Terminal.

Users can decide if they want to use the VM to store other documents/files or just use it to read Qubes documentation when they are offline.

---

Reply to this email directly or view it on GitHub:
#1019 (comment)

We've discussed this recently and concluded that we want to have HTML
version there - to have links working. So probably needs to do some
preprocessing on it: call yekyll in DispVM on it (download and install
it there first).
Then as you've said - create dedicated VM and place those files there
using management stack.

Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?

[marmarek]

On Fri, Sep 04, 2015 at 04:47:39PM -0700, Michael Carbone wrote:

I agree it should be part of a dedicated AppVM. Any updates on this @nrgaway ?

I think having a more general "offline-files" or "offline-docs" AppVM pre-populated with Qubes documentation would be awesome. Default displayed apps would probably be Help (going to qubes-doc content), Files, Terminal.

Users can decide if they want to use the VM to store other documents/files or just use it to read Qubes documentation when they are offline.

---

Reply to this email directly or view it on GitHub:
#1019 (comment)

We've discussed this recently and concluded that we want to have HTML
version there - to have links working. So probably needs to do some
preprocessing on it: call yekyll in DispVM on it (download and install
it there first).
Then as you've said - create dedicated VM and place those files there
using management stack.

Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?

[andrewdavidwong]

Are you sure it's feasible to have the local content render correctly in a browser without using jekyll serve?

If I do jekyll build, then open _site/index.html or _site/doc/[...]/index.html, it's pretty much unreadable. But if I do jekyll serve then navigate to http://127.0.0.1:4000/, the whole website functions perfectly.

[andrewdavidwong]

Are you sure it's feasible to have the local content render correctly in a browser without using jekyll serve?

If I do jekyll build, then open _site/index.html or _site/doc/[...]/index.html, it's pretty much unreadable. But if I do jekyll serve then navigate to http://127.0.0.1:4000/, the whole website functions perfectly.

[marmarek]

Whatever approach will be chosen, we still need to have just
documentation there. It would be weird to serve the whole website as an
offline doc...

Regarding using jekyll build vs jekyll serve I think having already
rendered pages is better than rendering them on demand. Mostly because
jekyll requirements - a lot of unsigned code. While we can render it
using DispVM and just have generated HTMLs, forcing user to use DispVM
every time he/she want to access documentation doesn't look like a good
idea (especially while searching how to fix problems with DispVMs...).
If accessing the files directly from browser wouldn't work, we can
always start some minimalistic http server which expose
http://127.0.0.1:4000/.

Anyway I see the point of using jekyll serve would be much easier...
So we can have this as "plan B" if the "plan A" will happen to be too
hard to implement.

PS I don't agree that jekyll-rendered HTMLs are unreadable - the most
complex parts are just from our templates. Anyway HTML isn't meant to be
read as is directly. And probably if someone want text-only version we
can have markdown files there also (in addition to HTML).

Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?

[marmarek]

Whatever approach will be chosen, we still need to have just
documentation there. It would be weird to serve the whole website as an
offline doc...

Regarding using jekyll build vs jekyll serve I think having already
rendered pages is better than rendering them on demand. Mostly because
jekyll requirements - a lot of unsigned code. While we can render it
using DispVM and just have generated HTMLs, forcing user to use DispVM
every time he/she want to access documentation doesn't look like a good
idea (especially while searching how to fix problems with DispVMs...).
If accessing the files directly from browser wouldn't work, we can
always start some minimalistic http server which expose
http://127.0.0.1:4000/.

Anyway I see the point of using jekyll serve would be much easier...
So we can have this as "plan B" if the "plan A" will happen to be too
hard to implement.

PS I don't agree that jekyll-rendered HTMLs are unreadable - the most
complex parts are just from our templates. Anyway HTML isn't meant to be
read as is directly. And probably if someone want text-only version we
can have markdown files there also (in addition to HTML).

Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?

[andrewdavidwong]

Regarding using jekyll build vs jekyll serve I think having already rendered pages is better than rendering them on demand. [...]

I definitely agree that just having pre-rendered pages would be better. (I wasn't trying to advocate using Jekyll for this. I was just wondering if/how it's feasible to do so without Jekyll, given how the documentation currently works.)

PS I don't agree that jekyll-rendered HTMLs are unreadable - the most complex parts are just from our templates.

Ok, then that's good news. :)

Anyway HTML isn't meant to be read as is directly.

Not sure I understand. Do you mean reading the HTML source directly, or "rendering" it in a browser? (I was just opening the HTML files in _site/ in Firefox after locally building the site with Jekyll.)

And probably if someone want text-only version we can have markdown files there also (in addition to HTML).

Right. Actually, I thought the original point of writing everything in Markdown was to make it easy to read the plain text source, so that the plain text Markdown source would simply be the offline documentation.

Nonetheless, I agree that it would be a further improvement to have some form of "pre-rendered" offline documentation in addition to the raw Markdown source, especially if our desiderata include things like working (internal) links and embedded images. (Anything else besides these?)

[andrewdavidwong]

Regarding using jekyll build vs jekyll serve I think having already rendered pages is better than rendering them on demand. [...]

I definitely agree that just having pre-rendered pages would be better. (I wasn't trying to advocate using Jekyll for this. I was just wondering if/how it's feasible to do so without Jekyll, given how the documentation currently works.)

PS I don't agree that jekyll-rendered HTMLs are unreadable - the most complex parts are just from our templates.

Ok, then that's good news. :)

Anyway HTML isn't meant to be read as is directly.

Not sure I understand. Do you mean reading the HTML source directly, or "rendering" it in a browser? (I was just opening the HTML files in _site/ in Firefox after locally building the site with Jekyll.)

And probably if someone want text-only version we can have markdown files there also (in addition to HTML).

Right. Actually, I thought the original point of writing everything in Markdown was to make it easy to read the plain text source, so that the plain text Markdown source would simply be the offline documentation.

Nonetheless, I agree that it would be a further improvement to have some form of "pre-rendered" offline documentation in addition to the raw Markdown source, especially if our desiderata include things like working (internal) links and embedded images. (Anything else besides these?)

[mfc]

call jekyll in DispVM on it (download and install it there first).

i might be mis-reading this (correct me if so), but are we assuming the user has internet access? I think the point of this ticket is to provide offline documentation at the installation stage in case the user does not have (or want) internet access.

maybe have it as a qubes package so that they can update the offline documentation if desired. alternatively it is only updated on new release candidates.

I agree that it would be a further improvement to have some form of "pre-rendered" offline documentation in addition to the raw Markdown source, especially if our desiderata include things like working (internal) links and embedded images.

agreed, this is for average users who may be put off by markdown formatting and who would benefit most from images being incorporated directly into pages, functional links, etc.

[mfc]

call jekyll in DispVM on it (download and install it there first).

i might be mis-reading this (correct me if so), but are we assuming the user has internet access? I think the point of this ticket is to provide offline documentation at the installation stage in case the user does not have (or want) internet access.

maybe have it as a qubes package so that they can update the offline documentation if desired. alternatively it is only updated on new release candidates.

I agree that it would be a further improvement to have some form of "pre-rendered" offline documentation in addition to the raw Markdown source, especially if our desiderata include things like working (internal) links and embedded images.

agreed, this is for average users who may be put off by markdown formatting and who would benefit most from images being incorporated directly into pages, functional links, etc.

[marmarek]

On Wed, Sep 23, 2015 at 02:05:45AM -0700, Axon wrote:

Not sure I understand. Do you mean reading the HTML source directly, or "rendering" it in a browser? (I was just opening the HTML files in _site/ in Firefox after locally building the site with Jekyll.)

Ah, I see :) Ok, so the above is still true - the ugly part is from our
template (nav bar etc), which doesn't render correctly in "offline"
version. We can have separate template for offline docs.

And probably if someone want text-only version we can have markdown files there also (in addition to HTML).

Right. Actually, I thought the original point of writing everything in Markdown was to make it easy to read the plain text source, so that the plain text Markdown source would simply be the offline documentation.

Yes, that's the initial idea. Also to be able to check the history, and
what you really sign, without using some external tools.
But for browsing the docs, simple text editor isn't enough because of
links not working...

Nonetheless, I agree that it would be a further improvement to have some form of "pre-rendered" offline documentation in addition to the raw Markdown source, especially if our desiderata include things like working (internal) links and embedded images. (Anything else besides these?)

I think that's the only reason for having HTML.

Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?

[marmarek]

On Wed, Sep 23, 2015 at 02:05:45AM -0700, Axon wrote:

Not sure I understand. Do you mean reading the HTML source directly, or "rendering" it in a browser? (I was just opening the HTML files in _site/ in Firefox after locally building the site with Jekyll.)

Ah, I see :) Ok, so the above is still true - the ugly part is from our
template (nav bar etc), which doesn't render correctly in "offline"
version. We can have separate template for offline docs.

And probably if someone want text-only version we can have markdown files there also (in addition to HTML).

Right. Actually, I thought the original point of writing everything in Markdown was to make it easy to read the plain text source, so that the plain text Markdown source would simply be the offline documentation.

Yes, that's the initial idea. Also to be able to check the history, and
what you really sign, without using some external tools.
But for browsing the docs, simple text editor isn't enough because of
links not working...

Nonetheless, I agree that it would be a further improvement to have some form of "pre-rendered" offline documentation in addition to the raw Markdown source, especially if our desiderata include things like working (internal) links and embedded images. (Anything else besides these?)

I think that's the only reason for having HTML.

Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?

[marmarek]

On Wed, Sep 23, 2015 at 02:23:21AM -0700, Michael Carbone wrote:

call jekyll in DispVM on it (download and install it there first).

i might be mis-reading this (correct me if so), but are we assuming the user has internet access? I think the point of this ticket is to provide offline documentation at the installation stage in case the user does not have (or want) internet access.

No, that was about generating HTMLs from Markdown during preparation of
such offline docs (during ISO build). Then the user will have just a set
of HTML files to browse.

maybe have it as a qubes package so that they can update the offline documentation if desired. alternatively it is only updated on new release candidates.

That's exactly the plan :)

Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?

[marmarek]

On Wed, Sep 23, 2015 at 02:23:21AM -0700, Michael Carbone wrote:

call jekyll in DispVM on it (download and install it there first).

i might be mis-reading this (correct me if so), but are we assuming the user has internet access? I think the point of this ticket is to provide offline documentation at the installation stage in case the user does not have (or want) internet access.

No, that was about generating HTMLs from Markdown during preparation of
such offline docs (during ISO build). Then the user will have just a set
of HTML files to browse.

maybe have it as a qubes package so that they can update the offline documentation if desired. alternatively it is only updated on new release candidates.

That's exactly the plan :)

Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?

[mfc]

awesome 👍

i'll get out the way :)

[mfc]

awesome 👍

i'll get out the way :)

[mfc]

looping in @bnvk in case he can provide any insight into this re: HTML, jekyll, etc

[mfc]

looping in @bnvk in case he can provide any insight into this re: HTML, jekyll, etc

[bnvk]

I'm going to take a stab at making offline docs, if that's ok with everyone else here! The approach I am doing is:

1. Design special theme for Docs similar, but different from normal website
2. Precompile the docs in Jekyll
3. Create a lightweight wrapper app using webkitgtk
4. Make this run / integrate with GNOME specific APIs

This is also a perfect place to integrate / plan for #1774 to be contained there in!

[bnvk]

I'm going to take a stab at making offline docs, if that's ok with everyone else here! The approach I am doing is:

1. Design special theme for Docs similar, but different from normal website
2. Precompile the docs in Jekyll
3. Create a lightweight wrapper app using webkitgtk
4. Make this run / integrate with GNOME specific APIs

This is also a perfect place to integrate / plan for #1774 to be contained there in!

[marmarek]

• Create a lightweight wrapper app using webkitgtk

Do you think it is really necessary? There is already a lot of HTML browsing applications...
Also take into account that we want it in separate VM (not dom0), so having such application probably would not make it any easier to integrate with GNOME

[marmarek]

• Create a lightweight wrapper app using webkitgtk

Do you think it is really necessary? There is already a lot of HTML browsing applications...
Also take into account that we want it in separate VM (not dom0), so having such application probably would not make it any easier to integrate with GNOME

[bnvk]

• Create a lightweight wrapper app using webkitgtk

Do you think it is really necessary? There is already a lot of HTML browsing applications...

There are web browsers... but launching / engaging with a bookmark on the desktop (or via start menu, like Whonix does), which opens a folder of HTML files in a web browser != a nice user experience.

To be clear, using webkitgtk+ wrapper should be a minimal amount of work. I'm not making a fully featured web browser just for our documentation, that would be silly. I'm proposing a nice launcher that behaves like a native app. The information architecture + navigation will still be generated by Jekyll and clicking around pages!

Alternatively, the official GNOME "help" API uses an XML based framework called mallard but that wouldn't allow us to use same docs on our website + would duplicate lots of work!

[bnvk]

• Create a lightweight wrapper app using webkitgtk

Do you think it is really necessary? There is already a lot of HTML browsing applications...

There are web browsers... but launching / engaging with a bookmark on the desktop (or via start menu, like Whonix does), which opens a folder of HTML files in a web browser != a nice user experience.

To be clear, using webkitgtk+ wrapper should be a minimal amount of work. I'm not making a fully featured web browser just for our documentation, that would be silly. I'm proposing a nice launcher that behaves like a native app. The information architecture + navigation will still be generated by Jekyll and clicking around pages!

Alternatively, the official GNOME "help" API uses an XML based framework called mallard but that wouldn't allow us to use same docs on our website + would duplicate lots of work!

[marmarek]

To be clear, using webkitgtk+ wrapper should be a minimal amount of work. I'm not making a fully featured web browser just for our documentation, that would be silly. I'm proposing a nice launcher that behaves like a native app. The information architecture + navigation will still be generated by Jekyll and clicking around pages!

Ok, if that's really that simple, and have that important impact on UX,
it may worth an effort.

Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?

[marmarek]

To be clear, using webkitgtk+ wrapper should be a minimal amount of work. I'm not making a fully featured web browser just for our documentation, that would be silly. I'm proposing a nice launcher that behaves like a native app. The information architecture + navigation will still be generated by Jekyll and clicking around pages!

Ok, if that's really that simple, and have that important impact on UX,
it may worth an effort.

Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?

[mfc]

what's the latest on this @bnvk?

[mfc]

what's the latest on this @bnvk?

[mfc]

3.2rc1 is coming up soon, it would amazing if we could include offline documentation in R3.2.

[mfc]

3.2rc1 is coming up soon, it would amazing if we could include offline documentation in R3.2.

[kalkin]

Have you ever considered using GNU info? It's pretty awesome and definitely underrated. You can export info pages as html, books or just use them at the command line or in Emacs.

[kalkin]

Have you ever considered using GNU info? It's pretty awesome and definitely underrated. You can export info pages as html, books or just use them at the command line or in Emacs.

[marmarek]

Sorry, I can't resist: https://xkcd.com/912/ ;)
Anyway standard info interface is quite hard to use, especially for someone not familiar with command line (which is also also a target for offline documentation).
We already have something that can be read directly (this is great in markdown - it is readable as plain text), can be rendered to html (using jekyll, or others) or some other format.

[marmarek]

Sorry, I can't resist: https://xkcd.com/912/ ;)
Anyway standard info interface is quite hard to use, especially for someone not familiar with command line (which is also also a target for offline documentation).
We already have something that can be read directly (this is great in markdown - it is readable as plain text), can be rendered to html (using jekyll, or others) or some other format.

[jpouellet]

It appears that the Qt Assistant might be a suitable offline help viewer similar to the hypothetical one described above, but that already exists, and also handles things like full-text search, user bookmarks, etc.

More information about the Qt Assistant help viewer itself can be found here:

• https://doc.qt.io/qt-5/assistant-quick-guide.html
• https://doc.qt.io/qt-5/assistant-details.html
• https://doc.qt.io/qt-5/assistant-custom-help-viewer.html

More information about the help format it uses (which appears to be just normal html+css + some extra index XML) can be found here:

• https://doc.qt.io/qt-5/qthelp-framework.html
• https://doc.qt.io/qt-5/qthelpproject.html

It also appears that the existing jekyll infrastructure could be re-used, just with a more minimal template without all the website header/footer boilerplate, and with a few extra files for the required index metadata. It would then be pretty simple to build and distribute the result as a package.

[jpouellet]

It appears that the Qt Assistant might be a suitable offline help viewer similar to the hypothetical one described above, but that already exists, and also handles things like full-text search, user bookmarks, etc.

More information about the Qt Assistant help viewer itself can be found here:

• https://doc.qt.io/qt-5/assistant-quick-guide.html
• https://doc.qt.io/qt-5/assistant-details.html
• https://doc.qt.io/qt-5/assistant-custom-help-viewer.html

More information about the help format it uses (which appears to be just normal html+css + some extra index XML) can be found here:

• https://doc.qt.io/qt-5/qthelp-framework.html
• https://doc.qt.io/qt-5/qthelpproject.html

It also appears that the existing jekyll infrastructure could be re-used, just with a more minimal template without all the website header/footer boilerplate, and with a few extra files for the required index metadata. It would then be pretty simple to build and distribute the result as a package.

[RefinedSoftwareLLC]

Offline Documentation, will allow Qubes OS to be one step closer to being usable by the majority, the normal people which will never learn to use a terminal. It is a real UI/UX issue, to not be able to view at least the FAQ before having to troubleshoot an internet connection issue. Keep up the good work with
making offline documentation a reality for Qubes OS!

[RefinedSoftwareLLC]

Offline Documentation, will allow Qubes OS to be one step closer to being usable by the majority, the normal people which will never learn to use a terminal. It is a real UI/UX issue, to not be able to view at least the FAQ before having to troubleshoot an internet connection issue. Keep up the good work with
making offline documentation a reality for Qubes OS!