Enable usage of external kallisto/bustools installation

[apeltzer]

Describe the issue

Hello!

@FloWuenne and I ran into an issue when trying to use a Singularity containerized version of this, which results in some shared library issues.

This is often the case, when precompiled binaries are shipped with the python package and is unfortunately true for the bioconda recipe for kb-python too. There is multiple things one could do:

a.) Patch around this, by simply patching out the kb-python function in config.py to "find" the linux/osx/windows binaries that are installed via bioconda
b.) allowing the config.py function to find / use externally managed bustools + kallisto versions too

I'm fairly thinking that b.) is the nicer option, but happy to be told otherwise - @dpryan79 and @Maarten-vd-Sande were also involved in some comments on this in the bioconda gitter, so I'm linking you in here too.

[apeltzer]

b.) could also be made in a way that allows "normal" users to run as before and more advanced ones who rely on installing the libraries e.g. in a container to assume that both tools are in the path. Would be a couple of lines of extra code without having to maintain patches.

[Lioscro]

Hi, @apeltzer,
I've also had this on my mind for some time now.
I was thinking adding an option to specify the command/path to a kallisto or bustools binary to use (instead of the provided one) with --kallisto or --bustools command-line arguments.

Would this work for your case?

[dpryan79]

The best implementation would be to make --kallisto and --bustools options truly optional and assume that the binaries are in your path unless they're specified.

[apeltzer]

As @dpryan79 is the more experienced here, this would be a good option I think - or a smart lookup functionality that simply checks for the existence of an external binary and if that fails uses the internal one?

[Lioscro]

The best implementation would be to make --kallisto and --bustools options truly optional and assume that the binaries are in your path unless they're specified.

Yeah, exactly. The arguments would be optional, though I think the provided binaries should be the default if those arguments are not provided.

Here is my proposed behavior.

• If --kallisto is provided, use that binary. Allow use of binaries in path, so if kallisto is a valid command, --kallisto kallisto would use the binary in your path.
• Otherwise, use provided binary.

Same for --bustools.

[Lioscro]

Related to discussion #107 (locked in favor of this issue).

[apeltzer]

Yup sounds good to me 👍🏻

[Lioscro]

@apeltzer @dpryan79 @Maarten-vd-Sande
This has been implemented in 7f4ae6c in devel. I've validated that it works on my end, but could some of you try it out as well?
If everything's working fine we can push a new release with these changes.
You can install from devel by running

pip install git+https://github.com/pachterlab/kb_python@devel

[Lioscro]

All changes in the devel branch are planned to be merged on Monday, April 12th, if there aren't any issues. A new version will be released to Pypi shortly after.

[Maarten-vd-Sande]

@Lioscro I am currently implementing this on the bioconda side. Does it work with any kallisto/bustools version? Or should I just pin the version you use now in kb-python?

0.40.0 for bustools
0.46.2 for kallisto

[Lioscro]

@Lioscro I am currently implementing this on the bioconda side. Does it work with any kallisto/bustools version? Or should I just pin the version you use now in kb-python?

0.40.0 for bustools
0.46.2 for kallisto

I didn't add a check for binary versions, but at a minimum kallisto bus should be implemented. Not sure when that was first introduced. (Additionally, bustools mash needs to be implemented for split indices.) To avoid the headache, I think it's fine to pin the versions to those included.

[github-actions]

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 5 days