Highly Variable and Degraded Performance from PyPi installation #110
Description
Describe the bug
Running calculations using xtb installed using pip leads to highly variable and often severely degraded performance (up to 3 orders of magnitude slower runtime). This performance is stochastic as calculations will sometimes take 3 orders of magnitude less time too!
To Reproduce
Create one conda environment for xtb:
conda create -n xtb xtb-python -c conda-forge Create a python environment for a pip install
python -m venv env
source ./env/bin/activate
pip install xtbRun the following script in each environment to compare timings. Run the script multiple time and you will see the performance of the pip install vary wildly from >.5 seconds to <.003 seconds.
import timeit
import numpy as np
from xtb.interface import Calculator, Param
from xtb.libxtb import VERBOSITY_FULL
numbers = np.array([8, 1, 1])
positions = np.array(
[
[0.0, 0.0, 0.0],
[0.524, 1.687, 0.480],
[1.146, -0.450, -1.354],
]
)
def calculate_xtb():
calc = Calculator(Param.GFN2xTB, numbers, positions)
calc.set_verbosity(VERBOSITY_FULL)
calc.set_output("logs.out")
calc.singlepoint()
# Number of times you want each function to be executed
number_of_executions = 10
# Time calculate_xtb
xtb_time = timeit.timeit(
"calculate_xtb()", globals=globals(), number=number_of_executions
)
print(
f"Average execution time for calculate_xtb: {xtb_time / number_of_executions:.5f} seconds"
)- What is your setup, which version are you running (current master is subject to change)
Latest conda and pip installs.
Timings in the conda environment will be stable around 0.003 seconds (on my laptop). Timings for the pip environment will vary from ~0.003 to >0.6 seconds.
Expected behaviour
The performance is the same for xtb regardless how it is installed. The performance will not vary over 3 orders of magnitude WITHIN the same install (i.e., the pip install will perform the same as the conda install on some executions and will perform 1000x slower on others. It appears random).
Additional context
The fact that the performance CAN be the same suggests something very funny is going on.