Loading imports gets stuck

[j-adamczyk]

I think there is a problem with imports in ogb. I tried to run the example:

import networkx as nx
print("a")
# from sklearn.metrics import roc_auc_score, average_precision_score
from ogb.graphproppred import PygGraphPropPredDataset
print("c")
from torch_geometric.loader import DataLoader

# Download and process data at './dataset/ogbg_molhiv/'
dataset = PygGraphPropPredDataset(name="ogbg-molhiv", root='dataset/')

split_idx = dataset.get_idx_split()
train_loader = DataLoader(dataset[split_idx["train"]], batch_size=32, shuffle=False)
valid_loader = DataLoader(dataset[split_idx["valid"]], batch_size=32, shuffle=False)
test_loader = DataLoader(dataset[split_idx["test"]], batch_size=32, shuffle=False)

a gets printed, but then Python gets stuck and does not print c. However I tried to add print("b") to various points in ogb, following the from ogb.graphproppred import PygGraphPropPredDataset line.

To import PygGraphPropPredDataset, we need to run the code in __init__.py:

from .evaluate import Evaluator
from .dataset import GraphPropPredDataset

try:
    from .dataset_pyg import PygGraphPropPredDataset
except ImportError:
    pass

try:
    from .dataset_dgl import DglGraphPropPredDataset
    from .dataset_dgl import collate_dgl
except (ImportError, OSError):
    pass

Even the first line does not get printed. So I went to the evaluate.py and there we have:

from sklearn.metrics import roc_auc_score, average_precision_score
print("b")

import pandas as pd
import os
import numpy as np

try:
    import torch
except ImportError:
    torch = None

And b does not get printed. So I added from sklearn.metrics import roc_auc_score, average_precision_score directly after print("a") in the original file. After this modification it works.

I had an opportunity to try this multiple times, on fresh installs of the entire OS (Windows 2 times, Linux 1 time), with three versions of Python (3.8, 3.9, 3.10) and 3 versions of PyTorch (3.9, 3.10, 3.11). The problem was identical every time, and every time adding the manual import worked.

Fixing this would be pretty nice, since this is a quickstart on the main page.

[weihua916]

Hi! This is a very weird bug, because this never happened to me, and it is not clear why your python cannot import sklearn inside the ogb package. I am using Ubuntu 16.4, Python 3.8

[j-adamczyk]

It definitely happens with:

• Windows 10
• Python 3.9.7
• ogb 1.3.3
• scikit-learn 1.0.2
• torch-geometric 2.0.5
• pytorch 1.10.2

Also with Ubuntu 20.04 (same versions). I killed the process after a good few minutes, so it's not just slow.

[weihua916]

Seems to be related to this issue: #322

[heavenlxj]

anyupdate here? i have got the same issue. the env listed here:
OS: CentOS Linux release 7.9.2009
Python: 3.9.12
ogb:1.3.3
sklearn: 1.0.2
pytorch: 1.8.0

i have checked the reply in the #322, i also found the interrupt will print the stacktrace, seems related to sklearn import, paste the stack here:

most recent call last):
File "/mnt/disk0/xjliu/pas-ogb/ogb-molhiv/extract_fingerprint.py", line 8, in
from ogb.graphproppred import GraphPropPredDataset
File "/root/miniconda3/envs/pas-test2/lib/python3.9/site-packages/ogb/graphproppred/init.py", line 1, in
from .evaluate import Evaluator
File "/root/miniconda3/envs/pas-test2/lib/python3.9/site-packages/ogb/graphproppred/evaluate.py", line 1, in
from sklearn.metrics import roc_auc_score, average_precision_score
File "/root/miniconda3/envs/pas-test2/lib/python3.9/site-packages/sklearn/init.py", line 82, in
from .base import clone
File "/root/miniconda3/envs/pas-test2/lib/python3.9/site-packages/sklearn/base.py", line 17, in
from .utils import _IS_32BIT
File "/root/miniconda3/envs/pas-test2/lib/python3.9/site-packages/sklearn/utils/init.py", line 25, in
from . import _joblib
File "/root/miniconda3/envs/pas-test2/lib/python3.9/site-packages/sklearn/utils/_joblib.py", line 7, in
import joblib
File "/root/miniconda3/envs/pas-test2/lib/python3.9/site-packages/joblib/init.py", line 113, in
from .memory import Memory, MemorizedResult, register_store_backend
File "/root/miniconda3/envs/pas-test2/lib/python3.9/site-packages/joblib/memory.py", line 32, in
from ._store_backends import StoreBackendBase, FileSystemStoreBackend
File "/root/miniconda3/envs/pas-test2/lib/python3.9/site-packages/joblib/_store_backends.py", line 15, in
from .backports import concurrency_safe_rename
File "/root/miniconda3/envs/pas-test2/lib/python3.9/site-packages/joblib/backports.py", line 7, in
from distutils.version import LooseVersion
File "", line 1007, in _find_and_load
File "", line 982, in _find_and_load_unlocked
File "", line 925, in _find_spec
File "/root/miniconda3/envs/pas-test2/lib/python3.9/site-packages/_distutils_hack/init.py", line 90, in find_spec
return method()
File "/root/miniconda3/envs/pas-test2/lib/python3.9/site-packages/_distutils_hack/init.py", line 101, in spec_for_distutils
mod = importlib.import_module('setuptools._distutils')
File "/root/miniconda3/envs/pas-test2/lib/python3.9/importlib/init.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "/root/miniconda3/envs/pas-test2/lib/python3.9/site-packages/setuptools/init.py", line 16, in
import setuptools.version
File "/root/miniconda3/envs/pas-test2/lib/python3.9/site-packages/setuptools/version.py", line 1, in
import pkg_resources
File "", line 211, in _lock_unlock_module
File "", line 107, in acquire
KeyboardInterrupt

Workaround:
same as the #322, i found it worked if i uninstalled the setuptools, but this will lead other runninng issue if missing this, so i hope someone can watch this, any help would be appreciated.

[heavenlxj]

provide some code repo, if someone need reproduce it on your env.
https://github.com/AutoML-Research/PAS-OGB

[j-adamczyk]

@heavenlxj error is still the same. The only workaround I found is adding from sklearn.metrics import roc_auc_score, average_precision_score before importing OGB.

[heavenlxj]

@j-adamczyk yep, it also worked for me, it's really helpful, thanks.

[Saydemr]

For me, the solution was using an older version of torch-geometric. Seems like the torch-geometric==1.7.0 works just fine. If torch-geometric version is more important try installing ogb==1.2.1

[rusty1s]

I just (hopefully) fixed this in #338. Let me know :)

[rajasbansal]

I got the same error with pytorch-geometric version 2.0.4. Updating pyg to 2.0.5 worked for me.

[HxyScotthuang]

Hi, I am still facing the exact same error, using ogb 1.3.4 and pytorch 1.8.2+cu111. I downgrade ogb 1.3.4 -> 1.2.1 as mentioned in the previous post and it works.

[Saydemr]

The bug should be resolved by version 1.3.5, you can check if upgrading the version to the latest one works.