Share the final conda environment.

[Marius1311]

Had to install a couple new packages, share the final version once we're done.

[michalk8]

Reminder to self - add R requirements file with all necessary packages.

[michalk8]

2 things:

• please remove the prefix in the environment.yaml file (last line) - it's specific to your paths
• give it a nicer name (e.g. cellrank_benchmarks) and make sure that once every NB has this env set as its kernel (so that I will get a message of unknown kernel if I haven't installed it previously)

[Marius1311]

Hi Mike, okay, let's do that.

[Marius1311]

Did you already try whether you can create a conda environment using this file?

[michalk8]

I've tried it just now, I can't, here's the message:

ResolvePackageNotFound: 
  - hdf5==1.10.6=nompi_haae91d6_101
  - readline==8.0=h0678c8f_2
  - libcxx==11.0.0=h439d374_0
  - pysocks==1.7.1=py38h5347e94_2
  - xz==5.2.5=haf1e3a3_1
  - metis==5.1.0=hb1e8313_1006
  - gmp==6.2.0=hb1e8313_3
  - ncurses==6.2=hb1e8313_2
  - freetype==2.10.3=ha233b18_0
  - libxml2==2.9.10=h2c6e4a5_2
  - gettext==0.19.8.1=h46ab8bc_1002
  - numpy==1.19.2=py38h8ccc501_1
  - sqlite==3.33.0=h960bd1c_1
  - chardet==3.0.4=py38h5347e94_1008
  - libiconv==1.16=haf1e3a3_0
  - scalapack==2.0.2=hb7119d5_1009
  - fontconfig==2.13.1=h79c0d67_1002
  - icu==67.1=hb1e8313_0
  - openssl==1.1.1h=haf1e3a3_0
  - cairo==1.16.0=ha8983da_1005
  - cffi==1.14.3=py38h9edaa1b_1
  - glib==2.66.1=h39b9ebd_0
  - mumps-mpi==5.2.1=hd105cc2_8
  - pycairo==1.20.0=py38h26abc9d_1
  - slepc4py==3.13.0=py38h71a7be8_2
  - brotlipy==0.7.0=py38h94c058a_1001
  - libffi==3.2.1=hb1e8313_1007
  - cryptography==3.1.1=py38h52adbb4_0
  - petsc4py==3.13.0=py38h026ade1_4
  - python==3.8.6=hcfdab8c_0_cpython
  - suitesparse==5.6.0=h0e59142_0
  - parmetis==4.0.3=hbc1d92b_1005
  - certifi==2020.6.20=py38h5347e94_2
  - hypre==2.18.2=hc9ba2bc_1
  - libopenblas==0.3.10=openmp_h63d9170_5
  - pixman==0.38.0=h01d97ff_1003
  - libgfortran==4.0.0=h50e675f_12
  - setuptools==49.6.0=py38h5347e94_2
  - tk==8.6.10=hb0a8c7a_1
  - libgfortran4==7.5.0=h50e675f_12
  - superlu_dist==6.2.0=h32cdba5_2
  - tbb==2019.9=ha1b3eb9_1
  - zlib==1.2.11=h7795811_1010
  - libpng==1.6.37=hb0a8c7a_2
  - llvm-openmp==11.0.0=h73239a0_1
  - pcre==8.44=hb1e8313_0
  - leidenalg==0.8.2=py38h869d7aa_1
  - ptscotch==6.0.9=h682e2fc_1
  - python-igraph==0.8.3=py38hcde0000_1
  - superlu==5.2.1=hc209f47_1207
  - slepc==3.13.4=h522eec7_0
  - petsc==3.13.6=h60ba9c1_0
  - scotch==6.0.9=h57311db_1

I think the issue is these parts =h57311db_1 and so on.
In the worst case, we can specify most of the packages (except for R or SLEPc/PETSc) as pip install in the yml. file.
Also, I'd like to keep this file minimal (I assume you just exported your current env and didn't prune it) - we can then later reuse this file for nbviewer, if desired.

[michalk8]

Ok, my hypothesis was correct - after removing that part, only libfortran was not resolved (I've removed it, since it doesn't seem necessary).
I've also removed the prefix and changed the name to cellrank_reproducibility, I hope you approve.
I've included cloudpickle and memory_profiler inside as well.
Could you please check if you can create the env?

[michalk8]

Re nbviewer - I'd reformat the table as follows:

• left columns will always link to the notebooks in our repo, right to nbviewer (only where is makes sense)
• the names are quite long, so in both columns, I'd only display only the figure number or shorten the names (i.e. only notebook name, not the relative path)

[Marius1311]

Hi Mike, I agree with the nbviewer and with shortening the name - not to just the figure number though, let's give the full notebook name.

[Marius1311]

Re the environment, I had to add quite a few more packages for the R stuff, will share with you later.

[michalk8]

@Marius1311 do you have the new environment? Does it include stuff I mention in #39 ?

[michalk8]

@Marius1311 I think CellRank and scVelo should be in the conda environment (especifally the latter since I think you're using specific commit). As for CellRank, nothing I did changes the defaults so 1.0.0 is fine from my side - I think from yours it should be as well, since in the plotting, you're using n_knots in GAM, which is officially out in 1.0.0.

[michalk8]

Re conda environment - I am also able to succesfully create it on the server (now I'm testing R package installation, seems to be working well so far). Stil think we need to include CellRank and scVelo inside.
I've added a README on how to create it, if you want, I can create a script that will validate if everything is installed and has the correct version.

[Marius1311]

Hi Mike, CellRank version 1.0.0 is fine from my side. As for scVelo, I run everything with 0.2.2, so feel free to include that.

[Marius1311]

We do not need a specific commit hash for scVelo, version 0.2.2 is fine.

[michalk8]

Thanks, done via theislab/cellrank_reproducibility@963be76

[Marius1311]

Hi @michalk8 , I'm sending you my final conda environment in slack.