[work on virtualisation] Current issues with virtualisation in macos

[servoz]

Following the last update of the virtualisation documentation it appears that anatomist is not available in mia. I open this ticket to help solve this.

[montigno]

Here is the error message:

from soma import aims
ImportError: cannot import name 'aims' from 'soma' (/home/brainvisa/Apps/MIA/soma-base/python/soma/__init__.py)

When we type the 'anatomist' command in the terminal, it works

[servoz]

I think we should check two things first:

• Can you uninstall soma-base? (and even all populse except mia_processes and popilse_mia). Under the hood, mia checks at startup if it is in a "development" or "user" state. From the result it loads the populse modules from various places. Then there is also the CASA_DISTRO, which will retrieve the other packages we need if they have not already been retrieved during the verification procedure. Except in special cases, the best is to install by ourselves only populse_mia and mia_processes (for the moment not yet in CASA_DISTRO).
  Tipically when I launch mia, the stdout is :

- Mia in "developer" mode
  . Using populse_mia package from /casa/home/Git_projects/populse_mia/python ...
    populse_mia version : 2.0.0-dev+5e8b3370
  . Using capsul package from /casa/host/src/capsul/master ...
  . Using soma package from /casa/host/build/python ...
  . Using soma_worflow package from /casa/host/build/python ...
  . Using populse_db package from /casa/host/src/populse/populse_db/master/python ...
  . Using mia_processes package from /casa/home/Git_projects/mia_processes/python ...
  . Using User_processes_ECdev package from /casa/home/Git_projects/populse_mia/processes ...
  . Using User_processes package from /casa/home/Git_projects/populse_mia/processes...

=> populse_mia and mia_processes packages in /casa/home/Git_projects/ (my dev directory), capsul, soma, soma_worflow, populse_db packages in /casa/host/ (CASA_DISTRO's)

• Secondly, you should check the version of python used for compilation in CASA_DISTRO.

Please, can you give:

• the command used to launch mia ?
• the stdout when you launch mia ?
• How does the bv_maker command work ?
• Can you find the conf/bv_maker.cfg file ?

[denisri]

Hm, I guess the problem stands from a double installation of soma-base.
The python aims module is installed as a sub-package of the main package soma (soma.aims), but here you seem to have reinstalled soma-base, which provides the main package soma. However soma.aims is not reinstalled in soma-base (soma-base is python-only, whereas aims is a C++/python binding).
When you are working with a developer VM installation, you should rather install source code using bv_maker, which already manages sources, compilation and installs in a consistent location.

[montigno]

I had installed soma-base in the same root repertory as populse_mia (by git clone).

I have remove it, and when I launch populse_mia, I have this new error message :

ImportError: version mismatch: you are running Python 3.7.11 but the soma.aims module was likely compiled for Python 2.

Mia is launched with this command : 'python3 populse_mia/python/populse_mia/main.py'
The bv_maker command does'nt work.
The bv_maker.cfg is located at /casa/install/share/casa-distro-3.0/distro/brainvisa/conf/bv_maker.cfg

[servoz]

Thanks.
So the first point is now fixed (double installation of soma-base) and i guess we have still to fix the second point (compilation with Python 3).
Can you past the /casa/install/share/casa-distro-3.0/distro/brainvisa/conf/bv_maker.cfg, please ?
You need access to bv_maker ! @denisri i feel a path definition issue. What do you think about it ?
@montigno, do you have bv_maker on your distrib ? (maybe in /casa/install/share/casa-distro-3.0/distro/brainvisa/bin ?)
for me:

$ which bv_maker
~/casa_distro/brainvisa-opensource-master/bin/bv_maker

[servoz]

First thing to do, get access to bv_maker.
Can you search where it is located on your distribution (locate ,find, etc.). I suspect a wong $PATH.

[denisri]

The docs in https://github.com/populse/populse_mia/blob/master/developer_doc/UsingVirtualisation.md seem OK to me, the explanation about switching to python3 is correct:

"add cmake_options += -DPYTHON_EXECUTABLE=/usr/bin/python3 in the [ build $CASA_BUILD ] section of the $HOME/casa_distro/brainvisa-opensource-master/conf/bv_maker.cfg file (host) to make python3 as default"

[montigno]

I have several bv_maker.cfg files:

/casa/install/share/casa-distro-3.0/distro/brainvisa/conf/bv_maker.cfg
/casa/install/share/casa-distro-3.0/distro/bvdev/conf/bv_maker.cfg
/casa/install/share/casa-distro-3.0/distro/cati_platform/conf/bv_maker.cfg
/casa/install/share/casa-distro-3.0/distro/cea/conf/bv_maker.cfg
/casa/install/share/casa-distro-3.0/distro/opensource/conf/bv_maker.cfg
/casa/install/share/casa-distro-3.0/distro/web/conf/bv_maker.cfg

I think the first is the good.

I added the line cmake_options in [ build $CASA_BUILD ] section, and I restarted the virtual machine : the problem is not solved.

To answer @servoz, I have not /casa/install/share/casa-distro-3.0/distro/brainvisa/bin .

[montigno]

I am on leave this evening and I will be back next Tueasday.
In the docs https://github.com/populse/populse_mia/blob/master/developer_doc/UsingVirtualisation.md, I added the installation procedure with SingularityCE Vagrant Box, you will see the error message when we run the 'bv' command.

Thanks.

[denisri]

No, none of them is the right one. These are template conf files, but one additional one should be found in $HOME/casa_distro/brainvisa-opensource-master/conf/ as explained in @servoz's doc. Isn't it here ?
(moreover you don't need to restart the VM)

[denisri]

Uh oh I think I understand. I had not fully read the doc on https://github.com/populse/populse_mia/blob/master/developer_doc/UsingVirtualisation.md, and it's wrong for the MacOS / VM case, the doc asks to install the image file "Brainvisa 5.0.2", which is a user image, already compiled for python2. Here you should install a developer image, as explained in: https://brainvisa.info/web/download.html#installing-a-virtualbox-developer-environment
The dev image allows to get the svn/git sources and rebuild everything, using either python2 or python3. The user image is not meant to develop and rebuild.

[servoz]

oh yes, that makes sense!!!
I think you have discovered the issue, @denisri!
@montigno will tell us, next week when he comes back from vacations.

[montigno]

Sorry for this lost time !

So, I imported and configured the developer image 'casa-dev-5.0-1.ova' in VirtualBox (like the user image).

Brainvisa and Anatomist don't seem to be installed, bv_maker doesn't work.
The searching of these items indicates that they exist in the folder /casa/bootstrap/brainvisa-cmake-5.0/cmake

The folders /casa/conf/svn.secret and /casa/conf/bv_maker.cfg (as explained in https://brainvisa.info/web/download.html#installing-a-virtualbox-developer-environment) don't exist.

What procedures should you follow now ? As if we were under Linux and follow the procedure https://github.com/populse/populse_mia/blob/master/developer_doc/UsingVirtualisation.md ?

Thanks for your help !

[servoz]

Brainvisa and Anatomist don't seem to be installed => I think this is normally before the first run of bv_maker
bv_maker doesn't work. => I think this is what we need to fix first.

You don't have bv_maker in /casa/bootstrap ? For me:
$ pwd
/casa/bootstrap
$ find . -name bv_maker
./brainvisa-cmake/bin/bv_maker

I think @denisri could give us some advice (he knows much better than I do about BV image architecture)

[servoz]

I think it can be useful.
That too.
And in a more general way.

[montigno]

yes I have bv_maker in /casa/bootstrap/brainvisa-cmake/bin, when I launch it I have this message :

Configuration file: None

bv_maker summary:
All went good.

I will look at the @servoz's link (How to compile BrainVISA projects)

[servoz]

I think it does not find a configuration file (bv_maker.cfg) ... This is the next step to fix

[denisri]

I took the time to try the vbox developer image, and it's outdated. I have opened a ticket:
brainvisa/casa-distro#272
It's possible to fix things manually using the current image, but we'd better build a new one (and fix things if needed)

[servoz]

Oh thanks @denisri for your message. That explains why @montigno had problems with this part!

Our goal, for the moment, is to evaluate how mia works (fluidity, heaviness) before proposing also the virtualisation for populse (and mia). If the correction of the image is fast, we might as well wait. If the manual corrections are fast enough, maybe we can already do them to continue our test phase?

The next step will be to modify the images to provide mia turnkey as it is the case for BV and Anatomist.
For this last part we will have to consult each other a bit I think (Do you agree that we should propose an image with BV, anatomist and mia? Why not include all populse, also mia_processes and populse_mia, in the bv_maker? etc. and etc. Anyway, I think we'll have to talk at some point about the global roadmap of our project).

But this last step will come when we will have finished the installation and operation tests.

[montigno]

thanks @denisri for this information.

So I wait for the next virtual image and in the meantime I continue to test the method singularity-vagrant.

[servoz]

I guess the issues for this this ticket are now fixed, right?
Please, @urlub , if that's the case, can you close that ticket?