New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update hpc_clusters_install.rst #1305

Closed
wants to merge 4 commits into
base: master
from
Jump to file or symbol
Failed to load files and symbols.
+15 −0
Diff settings

Always

Just for now

Next

Update hpc_clusters_install.rst

A few lines on installing in a different directory from the source.
  • Loading branch information...
levlafayette committed Apr 24, 2018
commit 21a5b8ab858efe427edfb96b1214256f2d0f1d0e
@@ -133,6 +133,21 @@ compiled on another system.
``gcc`` version must appear *before* that of the version installed on the
cluster in the ``LD_LIBRARY_PATH`` environment variable.
Installation of *MRtrix3* and dependencies in a new directory
-------------------------------------------------------------
In HPC systems and development environments is is extremely common and useful to have multiple versions of the same software available.
After following the installation process above, there will be new directories built in the source directory. Those used can be copied over to a preferred directory e.g.,
``mkdir -p /usr/local/MRtrix/3-3.0_RC3``
``cp -r bin/ lib/ share/ doc/ /usr/local/MRtrix/3-3.0_RC3``

This comment has been minimized.

@jdtournier

jdtournier May 22, 2018

Member

Only one comment here: there is no doc/ folder in the repo. Maybe you mean docs/, but that contains the source .rst files, rather than the formatted HTML. I'm not sure there's any value in copying those across.

If you do want to include the docs, we'd need to update the instructions to also run sphinx to build the docs - personally I think that's unnecessary, and adds additional dependencies. The easier option here is to download the corresponding version from readthedocs - e.g. this is the direct download for 3.0_RC3. We make sure all tagged releases get built, so the matching version should always be available. Otherwise, latest corresponds to the master branch, so assuming you download this at about the same time as the git clone or git pull, it should be right up to date. I think that would be a simpler approach if the docs also need to be installed.

What do you think?

This comment has been minimized.

@levlafayette

levlafayette May 23, 2018

I think it's much better to include the rst files than to use formatted html, which is an absolute nightmare to HPC admins and users to read.

This comment has been minimized.

@jdtournier

jdtournier May 24, 2018

Member

OK, I have to admit, this answer is not what I was expecting... I don't expect anyone to read these docs on the terminal, they're designed to be read in a browser outside the terminal session, and directly from our readthedocs site. This is why we don't ship them or provide instructions for building them - the RST sources are part of the repo to ensure consistency with the code, but they're not designed to necessarily be much use to end users as such.

Maybe you're after the reference pages for each individual command? We don't ship man pages, but each command will dump its help page if invoked without arguments (exception being mrview, but you can get that with an additional -help option). It dumps the same information as shown for that command on the docs site (e.g. here's the page for mrconvert). I suspect this would cover the bulk of users' needs directly on the command-line.

The rest of the docs are much more involved and expected to be used essentially for background reading and explaining complex concepts. There's a lot of cross-referencing going on, and I expect users will genuinely be much better served with the online HTML than any text-based version. The potential exception here might be installation, where I can see the appeal of having the instructions directly in the repo (on that front, I note the instructions on the README are somewhat out of date - I'll fix that now). But the docs are already directly available at install time, since you would've just cloned the repo, so that shouldn't be an issue.

So given all of this, I personally see no benefit in copying the docs/ folder over when deploying. There might be some rationale in copying the README though (once we fix it up...), since that does point to all the different sources of documentation and assistance, and provides instructions for upgrading - both of which might be useful post-install.

This comment has been minimized.

@levlafayette

levlafayette May 24, 2018

  1. Users on an HPC typically do not have access to a webbrowser on such systems. Providing html formatted documents for such people in that environment would be painful at best.

  2. Given that the docs also include installation instructions, HPC users (let alone sysadmins) may also wish to copy source files and do local installations within their own project directories for bespoke purposes. They will need to be able to read RST documents.

  3. As you say the docs directory also includes a lot of material that would often be found in man pages (e.g., command_line.rst etc)

Some of us really do spend 95% of our time on the command line. We may be almost exclusively in the HPC space, but it really should be such a big deal to make life easier for us.

This comment has been minimized.

@thijsdhollander

thijsdhollander May 24, 2018

Member

Hi Lev, fellow Melburnian here. 😉

Correct me if I'm wrong, but I've got the feeling some cross-purpose talk is going on here. Users on a HPC system would indeed not typically have a webbrowser on that HPC system available to them; but surely they would have a webbrowser available on the system that they use to access that HPC system, right? The latter is the system that @jdtournier was referring to; not the HPC system itself. I'm speaking here from the point of view of a lot of my/our collaborators in Melbourne for instance using the current Melbourne Bioinformatics cluster, or Monash' "MASSIVE" cluster. They're connecting to it via any SSH client on their own systems; quite often with (these days) a browser open next to it almost "by default" (even if only to read their email, so as to say). I can't imagine many users who would find themselves in a significantly different scenario...

For sysadmins, I'm not sure what their typical habits are. For them, I suppose, mostly the installation docs matter (as they won't be the users of the software, right?). I agree that at least a single text file, designed to be readable on the simplest text rendering display (e.g. the terminal) can be potentially useful. That would/should be the README file @jdtournier is referring to. We should make sure that's indeed nicely up to date, contains all the relevant information about the installation (and fixes for common problems with respect to that), and sits in a sensible location in the folder structure (top level, as where one would naturally expect to find the main README file for about any software).

I'm hoping this clarifies what may have been some confusion in the discussion above (that's at least my perception of it...)...?

Cheers,
Thijs

This comment has been minimized.

@levlafayette

levlafayette May 24, 2018

It is not always the case that users have access to a browser, especially if the user/admin is operating with a real terminal rather than a virtual terminal.

In any case system context switching is annoying. When operating in an environment is handy to have the relevant information in one's context. Text files are best, html formatting is horrible, and software developers who have their install and usage docs as PDF files should be tarred, feathered, and driven out of town.

Another issue (which I have encountered more than a couple of times) is when the installation and usage website for an application is down, or gone entirely. Websites don't last forever, but curiously researchers often like (for whatever reason) to use software that can be more than 20 years old.

I should also mention that sysadmins both install and (if they're worth their salt) do minimal test cases as well. They aren't experts in an application by any means, but after installation, they want to make sure it works.

In any case, it shouldn't be a big deal just to copy the docs to the suggested location. I am surprised that it has become so.

This comment has been minimized.

@jdtournier

jdtournier May 24, 2018

Member

This isn't a big deal, and doesn't need to become one. Clearly you have very strong opinions on the topic, and we have different opinions. We don't accept merge requests (external or internal) without questioning the rationale, which is why we're discussing this with you.

The point here is that these instructions should apply to the broadest set of users. These instructions are specific to HPC users, so I want to make sure the recommendations we give are appropriate to most HPC sites. MRtrix has been around for 10 years and been installed on many HPC sites, and this is the first time anyone has ever suggested this. While you may think it's a no-brainer, if we provide instructions to copy over the docs/, this carries with it our implicit recommendation that this is where users should look first. I don't think this is a good recommendation for most HPC sites, hence why I'm questioning it.

The only possible situation where I see these docs being of any values whatsoever is for users operating on pure terminals, with no possibility of running a browser. That's a very long way from our typical user base - I've never come across any users accessing a HPC system to use MRtrix from a real terminal, primarily for the reason that they do eventually need to display their results. So while such users may exist, I don't think that the docs needs to cater for such a rare use case, consisting of presumably very capable users who would be perfectly capable of making their own copy if they felt like it.

So I suggest we amend the README with clearer and up to date instructions, remove the docs/ folder from the initial cp, and add a subsequent section to this page about the documentation, with recommendations about where to find the docs online, how to download them locally if needed, and suggest a local copy of the docs/ folder for those rare cases where it might make sense.

This comment has been minimized.

@levlafayette

levlafayette May 24, 2018

"The only possible situation where I see these docs being of any values whatsoever is
for users operating on pure terminals, with no possibility of running a browser."

Of any values[sic]? Are you sure? Are you overlooking the reasons just provided? Are you overlooking the design principle of start with the basic principle of presenting data from the universal readable format (i.e., text) and then adding markup?

Still, if you're determined to make the application less helpful to HPC users and administrators, go right ahead.

Users will then need to export paths as appropriate. Most HPC systems will use an evironment modules system.
Remote display
--------------
ProTip! Use n and p to navigate between commits in a pull request.