pyGREAT gives error: AttributeError: 'NoneType' object has no attribute 'text

[MubasherMohammed]

Hi, thanks for developing this great package!
I am facing error when executing result = pyGREAT(region_sets, 'hg38', n_cpu=35, _temp_dir = tmp).

File ~/miniconda3/envs/scenicplus/lib/python3.8/site-packages/pycisTopic/pyGREAT.py:79, in pyGREAT(region_sets, species, rule, span, upstream, downstream, two_distance, one_distance, include_curated_reg_doms, bg_choice, tmp_dir, n_cpu, **kwargs)
77 ray.init(num_cpus=n_cpu, **kwargs)
78 sys.stderr = null
---> 79 pyGREAT_dict = ray.get(
80 [
81 pyGREAT_oneset_ray.remote(
82 region_sets[key],
83 species,
84 rule,
85 span,
86 upstream,
87 downstream,
88 two_distance,
89 one_distance,
90 include_curated_reg_doms,
91 bg_choice,
92 tmp_dir,
93 )
94 for key in region_sets.keys()
95 ]
96 )
97 ray.shutdown()
98 sys.stderr = sys.stderr

File ~/miniconda3/envs/scenicplus/lib/python3.8/site-packages/ray/_private/auto_init_hook.py:22, in wrap_auto_init..auto_init_wrapper(*args, **kwargs)
19 @wraps(fn)
20 def auto_init_wrapper(*args, **kwargs):
21 auto_init_ray()
---> 22 return fn(*args, **kwargs)

File ~/miniconda3/envs/scenicplus/lib/python3.8/site-packages/ray/_private/client_mode_hook.py:103, in client_mode_hook..wrapper(*args, **kwargs)
101 if func.name != "init" or is_client_mode_enabled_by_default:
102 return getattr(ray, func.name)(*args, **kwargs)
--> 103 return func(*args, **kwargs)

File ~/miniconda3/envs/scenicplus/lib/python3.8/site-packages/ray/_private/worker.py:2624, in get(object_refs, timeout)
2622 worker.core_worker.dump_object_store_memory_usage()
2623 if isinstance(value, RayTaskError):
-> 2624 raise value.as_instanceof_cause()
2625 else:
2626 raise value

RayTaskError(AttributeError): ray::pyGREAT_oneset_ray() (pid=1990234, ip=192.168.0.6)
File "/home/mubasher/miniconda3/envs/scenicplus/lib/python3.8/site-packages/pycisTopic/pyGREAT.py", line 174, in pyGREAT_oneset_ray
return pyGREAT_oneset(
File "/home/mubasher/miniconda3/envs/scenicplus/lib/python3.8/site-packages/pycisTopic/pyGREAT.py", line 278, in pyGREAT_oneset
jobId = soup.find("div", attrs={"class": "job_desc_info"}).text
AttributeError: 'NoneType' object has no attribute 'text'

I'm not sure exactly how to fix this error. my pycisTopic version '1.0a0' as am still using the old SCENIC+ version.
my region_set looks:
{'Topic1': +--------------+-----------+-----------+
| Chromosome | Start | End |
| (category) | (int64) | (int64) |
|--------------+-----------+-----------|
| chr1 | 145075000 | 145075500 |
| chr1 | 55740500 | 55741000 |
| chr1 | 200379000 | 200379500 |
| chr1 | 145075500 | 145076000 |
| ... | ... | ... |
| chrX | 68724500 | 68725000 |
| chrX | 46989000 | 46989500 |
| chrX | 117250500 | 117251000 |
| chrX | 73640000 | 73640500 |
+--------------+-----------+-----------+
Unstranded PyRanges object has 482,334 rows and 3 columns from 23 chromosomes.
For printing, the PyRanges was sorted on Chromosome.,
'Topic2': +--------------+-----------+-----------+
| Chromosome | Start | End |
| (category) | (int64) | (int64) |
|--------------+-----------+-----------|
| chr1 | 244816000 | 244816500 |
| chr1 | 110881500 | 110882000 |
| chr1 | 17896000 | 17896500 |
| chr1 | 160032000 | 160032500 |
| ... | ... | ... |
| chrX | 39462500 | 39463000 |
| chrX | 47510500 | 47511000 |
| chrX | 152765500 | 152766000 |
| chrX | 52964500 | 52965000 |
+--------------+-----------+-----------+
Unstranded PyRanges object has 52,355 rows and 3 columns from 23 chromosomes.
For printing, the PyRanges was sorted on Chromosome.,
'Topic3': +--------------+-----------+-----------+

any tips will be highly appreciated!
thanks..

[SeppeDeWinter]

Hi @MubasherMohammed

Thank you for opening the issue!
This function needs to be updated indeed.

I have added it to my todo list and will try to get to it.

Best,

Seppe

[MubasherMohammed]

Thank you @SeppeDeWinter that would be great!