Cannot import Dianna directly in python on Mac M1

[laurasootes]

Whenever I start python from the terminal on my Mac M1 and try to import Dianna, I get a message saying:

"OMP: Error #15: Initializing libomp.dylib, but found libomp.dylib already initialized.
OMP: Hint This means that multiple copies of the OpenMP runtime have been linked into the program. That is dangerous, since it can degrade performance or cause incorrect results. The best thing to do is to ensure that only a single OpenMP runtime is linked into the process, e.g. by avoiding static linking of the OpenMP runtime in any library. As an unsafe, unsupported, undocumented workaround you can set the environment variable KMP_DUPLICATE_LIB_OK=TRUE to allow the program to continue to execute, but that may cause crashes or silently produce incorrect results. For more information, please see http://openmp.llvm.org/
zsh: abort python"

I can execute python scripts from the terminal though that import dianna, so I'm not sure why this happens

[laurasootes]

interesting: I do not get the error if I change the order of imports like this

from dianna.utils.tokenizers import SpacyTokenizer
import dianna

[loostrum]

After some testing, we come to the following conclusions:

• This issues seems Mac-specific, but not M1-specific.
• It only happens with a conda install of python (pyenv works fine)
• The culprit within DIANNA is the spacy package. If not importing that, DIANNA doesn't crash. import spacy in a clean python interpreter does crash

See also this stackoverflow issue

[loostrum]

When installing python with conda, but all dianna dependencies with pip, there is no error.
I looked at the spacy dependencies, one of which is numpy which is a good candidate to be using openmp.
So I removed numpy, and reinstalled with conda instead. Then the error is back.

Per the stackoverflow link above, apparently we need to make sure that a version of numpy that doesn't use mkl is installed.
This is achieved by adding the nomkl package to conda, which then should make sure that the blas-linked version of numpy is installed instead of the mkl version. I haven't been able to get this to work yet though, just installing nomkl and then numpy results in a blas-linked numpy, but the error is still there.

[loostrum]

Even when installing all dependencies through conda, with nomkl, the error is still there. The only workarounds so far are:

• install python in a clean conda env, then install dianna + all dependencies through pip
• Set the environment variable mentioned in the error message, but do note the warning: As an unsafe, unsupported, undocumented workaround you can set the environment variable KMP_DUPLICATE_LIB_OK=TRUE to allow the program to continue to execute, but that may cause crashes or silently produce incorrect results.. The dianna tests do pass after setting this. Perhaps we could add a "known issues" section to the documentation and explain the possible workarounds there.