issues in running bonito

[dcopetti]

Hello,

I was able to install bonito in an environment with python 3.6.9, then if I run it I get:

$ bonito basecaller
Traceback (most recent call last):
  File "/home/copettid/anaconda3/envs/py36/bin/bonito", line 5, in <module>
    from bonito import main
  File "/home/copettid/anaconda3/envs/py36/lib/python3.6/site-packages/bonito/__init__.py", line 7, in <module>
    from bonito import train, tune
  File "/home/copettid/anaconda3/envs/py36/lib/python3.6/site-packages/bonito/tune.py", line 31, in <module>
    import optuna
  File "/home/copettid/anaconda3/envs/py36/lib/python3.6/site-packages/optuna/__init__.py", line 1, in <module>
    from optuna import dashboard  # NOQA
  File "/home/copettid/anaconda3/envs/py36/lib/python3.6/site-packages/optuna/dashboard.py", line 23, in <module>
    import optuna.study
  File "/home/copettid/anaconda3/envs/py36/lib/python3.6/site-packages/optuna/study.py", line 13, in <module>
    import pandas as pd  # NOQA
  File "/home/copettid/anaconda3/envs/py36/lib/python3.6/site-packages/pandas/__init__.py", line 55, in <module>
    from pandas.core.api import (
  File "/home/copettid/anaconda3/envs/py36/lib/python3.6/site-packages/pandas/core/api.py", line 29, in <module>
    from pandas.core.groupby import Grouper, NamedAgg
  File "/home/copettid/anaconda3/envs/py36/lib/python3.6/site-packages/pandas/core/groupby/__init__.py", line 1, in <module>
    from pandas.core.groupby.generic import DataFrameGroupBy, NamedAgg, SeriesGroupBy
  File "/home/copettid/anaconda3/envs/py36/lib/python3.6/site-packages/pandas/core/groupby/generic.py", line 56, in <module>
    import pandas.core.algorithms as algorithms
AttributeError: module 'pandas' has no attribute 'core'

Can you help me figure out the type of issue I am having here?

Also, my fast5 files are in a 1.8 TB file.tar archive: do I need to extract them from there or can I use this archive as an input directly?
Thanks,
Dario

[iiSeymour]

Hey @dcopetti

It looks like you are using Anaconda which is not something I use myself. This StackOverflow link has a few suggestions and I suspect it's a conflict between using mix of conda and pip. What do you get if you run the following two commands?

$ conda list | grep pandas
$ python3 -m pip list | grep pandas

The pandas dependency comes from optuna which is used only for bonito tune and is not a requirement for basecalling so a quick workaround would be to remove the import of tune from bonito/__init__.py.

Yes, you will need to unpack the reads from the .tar before you can basecall.

HTH,

Chris.

[dcopetti]

Hi Chris,

Thank you for the feedback.
The two commands result in:

(py36) bash-4.2$ python --version
Python 3.6.9 :: Anaconda, Inc.
(py36) bash-4.2$ conda list | grep pandas
pandas                    1.0.3            py36h0573a6f_0
(py36) bash-4.2$ python3 -m pip list | grep pandas
pandas                             1.0.3

I tried removing the import of tune:

try:
    from bonito import train
    modules.extend(['train'])
except ImportError:
    pass

and for now it works:
(py36) bash-4.2$ bonito basecaller
usage: bonito basecaller [-h] [--device DEVICE] [--weights WEIGHTS]
[--beamsize BEAMSIZE] [--half]
model_directory reads_directory
bonito basecaller: error: the following arguments are required: model_directory, reads_directory
however, when I run it I get this error:

(py36) bash-4.2$ bonito basecaller dna_r9.4.1 reads2/
> loading model
Traceback (most recent call last):
  File "/home/copettid/anaconda3/envs/py36/bin/bonito", line 8, in <module>
    sys.exit(main())
  File "/home/copettid/anaconda3/envs/py36/lib/python3.6/site-packages/bonito/__init__.py", line 39, in main
    args.func(args)
  File "/home/copettid/anaconda3/envs/py36/lib/python3.6/site-packages/bonito/basecaller.py", line 19, in main
    model = load_model(args.model_directory, args.device, weights=int(args.weights), half=args.half)
  File "/home/copettid/anaconda3/envs/py36/lib/python3.6/site-packages/bonito/util.py", line 186, in load_model
    model.to(device)
  File "/home/copettid/anaconda3/envs/py36/lib/python3.6/site-packages/torch/nn/modules/module.py", line 425, in to
    return self._apply(convert)
  File "/home/copettid/anaconda3/envs/py36/lib/python3.6/site-packages/torch/nn/modules/module.py", line 201, in _apply
    module._apply(fn)
  File "/home/copettid/anaconda3/envs/py36/lib/python3.6/site-packages/torch/nn/modules/module.py", line 201, in _apply
    module._apply(fn)
  File "/home/copettid/anaconda3/envs/py36/lib/python3.6/site-packages/torch/nn/modules/module.py", line 201, in _apply
    module._apply(fn)
  [Previous line repeated 3 more times]
  File "/home/copettid/anaconda3/envs/py36/lib/python3.6/site-packages/torch/nn/modules/module.py", line 223, in _apply
    param_applied = fn(param)
  File "/home/copettid/anaconda3/envs/py36/lib/python3.6/site-packages/torch/nn/modules/module.py", line 423, in convert
    return t.to(device, dtype if t.is_floating_point() else None, non_blocking)
  File "/home/copettid/anaconda3/envs/py36/lib/python3.6/site-packages/torch/cuda/__init__.py", line 196, in _lazy_init
    _check_driver()
  File "/home/copettid/anaconda3/envs/py36/lib/python3.6/site-packages/torch/cuda/__init__.py", line 101, in _check_driver
    http://www.nvidia.com/Download/index.aspx""")
AssertionError:
Found no NVIDIA driver on your system. Please check that you
have an NVIDIA GPU and installed a driver from
http://www.nvidia.com/Download/index.aspx

do I need to have GPUs to run bonito?
From running lshw -short I get this:

                     system     Computer
/0                   bus        Motherboard
/0/0                 memory     976GiB System memory
/0/1                 processor  Intel(R) Xeon(R) Platinum 8168 CPU @ 2.70GHz

I have 48 CPUs in this virtual machine
Thanks,
Dario

[iiSeymour]

Bonito can run on the CPU with --device cpu however it's more for validation and I wouldn't recommend calling a large batch of reads on the CPU as PyTorch will only use a single core. Even heavily optimized CPU runtimes that use all cores are significantly slower than a single GPU so my recommendation would be to run on GPU if you can. Also, see #10.

[dcopetti]

thanks.
At this point I would switch to another basecaller that I can run on my own with CPUs:
is Guppy the best out there now? If so, where can I download it? I don't find it in Github or in the ONT community pages

[iiSeymour]

Guppy sounds like the best option at this time, all downloads are here.