Can I help update the Bioconda package?

[pinin4fjords]

Hi!

I'd like to use the most up-to-date version of basejump for aggregating single-cell data. We're very Bioconda-centric, can I help update the Bioconda recipe? I just tried the dummy way of simply replacing the package source but that didn't work due to dependencies on your bioverbs etc- do they need their own Bioconda packages? If you give me a rough idea of what needs to be done I'll have another go.

Thanks,

Jon

[mjsteinbaugh]

Hi Jon, sure I'd appreciate the help! That's correct that the updated dependencies for v0.9 (e.g. bioverbs) need to also be on bioconda, which is why the v0.7 release is currently pinned there. I'm in the process of finishing up the documentation on the package and submitting to Bioconductor soon. Are you also using bcbioSingleCell, or just some of the functions that work on SingleCellExperiment inside basejump? If so, I'll work on getting bcbioSingleCell up on bioconda also.

[mjsteinbaugh]

For reference, the bioconda recipe is here:
https://github.com/bioconda/bioconda-recipes/tree/master/recipes/r-basejump

[pinin4fjords]

I'm mostly interested in the aggregation functions so I can collapse some technical replicates. Happy spend the time helping here rather than rolling my own. So shall I go ahead and build recipes for your dependencies?

[mjsteinbaugh]

These are the new dependencies that need recipes for basejump v0.9:

• bioverbs
• brio
• goalie
• syntactic
• transformer

Everything should still be pinned against R 3.4 to work with bioconda. I split some of the functions in basejump out into these subpackages, making it a lot easier to unit test.

[pinin4fjords]

Okay cool, I'll try those and ping you for reviews.

[pinin4fjords]

Having trouble building goalie- I'm a little ignorant of the mechanisms of that package. Do you know what's needed in Bioconda to resolve the following?

...
-FORTRANFLAGS=-march=nocona -mtune=core2 -ftree-vectorize -fPIC -fstack-protector -O2 -pipe -I$PREFIX/include -fdebug-prefix-map=$SRC_DIR=/usr/local/src/conda/r-goalie-0.2.6 -fdebug-prefix-map=$PREFIX=/usr/local/src/conda-prefix
-GFORTRAN=/Users/jmanning/miniconda3/bin/x86_64-apple-darwin13.4.0-gfortran
* installing to library ‘/Users/jmanning/miniconda3/conda-bld/r-goalie_1549898449299/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehol/lib/R/library’
* installing *source* package ‘goalie’ ...
** R
** preparing package for lazy loading
Error in is.function(fun) : could not find function "...elt"
Error : unable to load R code in package ‘goalie’
ERROR: lazy loading failed for package ‘goalie’
* removing ‘/Users/jmanning/miniconda3/conda-bld/r-goalie_1549898449299/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehol/lib/R/library/goalie’
Traceback (most recent call last):
  File "/Users/jmanning/miniconda3/bin/conda-build", line 11, in <module>
    sys.exit(main())
  File "/Users/jmanning/miniconda3/lib/python3.7/site-packages/conda_build/cli/main_build.py", line 456, in main
    execute(sys.argv[1:])
  File "/Users/jmanning/miniconda3/lib/python3.7/site-packages/conda_build/cli/main_build.py", line 447, in execute
    verify=args.verify, variants=args.variants)
  File "/Users/jmanning/miniconda3/lib/python3.7/site-packages/conda_build/api.py", line 208, in build
    notest=notest, need_source_download=need_source_download, variants=variants)
  File "/Users/jmanning/miniconda3/lib/python3.7/site-packages/conda_build/build.py", line 2311, in build_tree
    notest=notest,
  File "/Users/jmanning/miniconda3/lib/python3.7/site-packages/conda_build/build.py", line 1477, in build
    cwd=src_dir, stats=build_stats)
  File "/Users/jmanning/miniconda3/lib/python3.7/site-packages/conda_build/utils.py", line 374, in check_call_env
    return _func_defaulting_env_to_os_environ('call', *popenargs, **kwargs)
  File "/Users/jmanning/miniconda3/lib/python3.7/site-packages/conda_build/utils.py", line 354, in _func_defaulting_env_to_os_environ
    raise subprocess.CalledProcessError(proc.returncode, _args)
subprocess.CalledProcessError: Command '['/bin/bash', '-e', '/Users/jmanning/miniconda3/conda-bld/r-goalie_1549898449299/work/conda_build.sh']' returned non-zero exit status 1.

[pinin4fjords]

Never mind, got it

[mjsteinbaugh]

Weird yeah ...elt() is part of base R so that shouldn't be an issue. Internally that function is used inside of assert() similar to the approach in base stopifnot().

[pinin4fjords]

I /think/ it wasn't available before 3.4.2, and the Bioconda's 3.4 is 3.4.1. I think we need to use r-backports, and call backports::...elt . Some judicious sed in build.sh I think?

[mjsteinbaugh]

Oh nice I think you're right and I didn't pick that up in my build checks. I'll work on adding that for R 3.4.1 compatibility.

[pinin4fjords]

Okay good. The sed in build.sh works for now. I'll wait to push the recipe until the naming thing is resolved on bioconda/bioconda-recipes#13592

[pinin4fjords]

Can't put this on Bioconda yet because of the CI issues with its dependencies, but if you'd like to check the transformer recipe: https://github.com/bioconda/bioconda-recipes/compare/master...ebi-gene-expression-group:r-transformer?expand=1. Are the dependencies appropriate? It does build (using our group Conda channel temporarily for the dependencies).

[pinin4fjords]

I've done all the recipes now, will progressively PR to Bioconda (CI fixed now)

[mjsteinbaugh]

Thanks @pinin4fjords I finished pushing changes that ensure R 3.4 compatibility, so this should be good to go on bioconda.

[pinin4fjords]

Thanks @mjsteinbaugh . The 3.4.1 issue went away when we stopped pinning (default is now 3.5.1).

[pinin4fjords]

(though obviously you're right- people who need to use 3.4 along with your stuff can)

[mjsteinbaugh]

Okay great, r-basejump should be ready to update on bioconda.

However, one thing we need to confirm first is that the recipes for r-bcbiobase and r-bcbiornaseq don't get messed up by the update. Currently, these versions on bioconda require basejump v0.7.2, but aren't fully compatible with the v0.9 update, so we should ensure that this is pinned in those recipes for the time being. These packages are used inside of bcbio-nextgen as part of the RNA-seq pipeline, so we need to be conservative with the updates to these dependencies on bioconda.

Currently, it looks like the r-bcbiobase is only requiring r-basejump >= 0.7.2, whereas I think should we should pin it specifically to 0.7.2 for the time being.

Recipe links:

• r-bcbiobase
• r-bcbiornaseq

[pinin4fjords]

I'm afraid I'm tied up with other things now. Maybe you could sort those? I do have the basejump recipe ready to go when you're ready.

[mjsteinbaugh]

I'll update those recipes. Thanks for the help @pinin4fjords

[mjsteinbaugh]

Updated with this pull request:
bioconda/bioconda-recipes#13632