Interface_analyser memory problem

[hbat1]

interface_ctc_analyzer.py prot1.pdb run1.xtc run2.xtc run3.xtc run4.xtc --frag_idxs_group_1 0,1 --frag_idxs_group_2 2,3 --short_AAs --t_unit mus --sort_by_av_ctcs --curve_color Hobat --graphic_dpi 300 --graphic_ext png --n_jobs 8 --n_smooth_hw 20 --BW_uniprot adrb2_human --CGN 3SN6 --ctc_cutoff_Ang 4
Will compute contact frequencies for the files:
run1.xtc
run2.xtc
run3.xtc
run4.xtc
with a stride of 1 frames.

Auto-detected fragments with method resSeq
fragment 0 with 203 AAs GLU30( 0)-LYS232 (202 ) (0)
fragment 1 with 78 AAs PHE264( 203)-CYSP341(280 ) (1)
fragment 2 with 69 AAs CYSP2( 281)-GLY70 (349 ) (2)
fragment 3 with 310 AAs ASP85( 350)-LEU394 (659 ) (3)
fragment 4 with 339 AAs SER2( 660)-ASN340 (998 ) (4)
fragment 5 with 67 AAs ALA2( 999)-CYSG68 (1065) (5)
fragment 6 with 3 AAs P0G395(1066)-MG397 (1068) (6)
requesting https://gpcrdb.org/services/residues/extended/adrb2_human ...done!
BW-numbering aligns best with fragments: [0, 1] (first-last: GLU30-CYSP341).
Input alternative in a format 1,2-6,10,20-25 or hit enter

5
166 ASN196 5.35 5.35 1
167 GLN197 5.36 5.36 1
168 ALA198 5.37 5.37 1
169 TYR199 5.38 5.38 1
170 ALA200 5.39 5.39 1
171 ILE201 5.40 5.40 1
172 ALA202 5.41 5.41 1
173 SER203 5.42 5.42 1
174 SER204 5.43 5.43 1
175 ILE205 5.44 5.44 1
176 VAL206 5.45 5.45 1
177 SER207 5.46 5.46 1
178 PHE208 5.47 5.47 1
179 TYR209 5.48 5.48 1
180 VAL210 5.49 5.49 1
181 PRO211 5.50 5.50 1
182 LEU212 5.51 5.51 1
183 VAL213 5.52 5.52 1
184 ILE214 5.53 5.53 1
185 MET215 5.54 5.54 1
186 VAL216 5.55 5.55 1
187 PHE217 5.56 5.56 1
188 VAL218 5.57 5.57 1
189 TYR219 5.58 5.58 1
190 SER220 5.59 5.59 1
191 ARG221 5.60 5.60 1
192 VAL222 5.61 5.61 1
193 PHE223 5.62 5.62 1
194 GLN224 5.63 5.63 1
195 GLU225 5.64 5.64 1
196 CYS226 None 5.65 1
197 LYS227 5.66 5.66 1
198 ARG228 5.67 5.67 1
199 GLN229 5.68 5.68 1
200 LEU230 5.69 5.69 1
201 GLN231 5.70 5.70 1

The consensus was kept, I am relabelling these:
5.65

INFO: these are the BW fragments mapped onto your topology
TM6 with 37 AAs LYS232( 202)-GLN299 (238 ) (TM6) 6.25-6.61
more than 1 fragments present. Input the ones to keep [0 1]0
TM1 with 32 AAs GLU30( 0)-PHE61 (31 ) (TM1)
ICL1 with 4 AAs GLU62( 32)-GLN65 (35 ) (ICL1)
TM2 with 31 AAs THR66( 36)-MET96 (66 ) (TM2)
ECL1 with 4 AAs MET98( 68)-PHE101 (71 ) (ECL1)
TM3 with 36 AAs GLY102( 72)-SER137 (107 ) (TM3)
ICL2 with 8 AAs PRO138( 108)-LEU145 (115 ) (ICL2)
TM4 with 27 AAs THR146( 116)-HIS172 (142 ) (TM4)
ECL2 with 3 AAs CYS191( 161)-PHE193 (163 ) (ECL2)
TM5 with 36 AAs ASN196( 166)-GLN231 (201 ) (TM5)
TM6 with 1 AAs LYS232( 202)-LYS232 (202 ) (TM6)
TM7 with 25 AAs ARG304( 243)-ARG328 (267 ) (TM7)
H8 with 11 AAs SER329( 268)-LEU339 (278 ) (H8)
Hit enter to continue!

CGN-numbering aligns best with fragments: [2, 3] (first-last: CYSP2-LEU394).
Input alternative in a format 1,2-6,10,20-25 or hit enter

H.hehf
452 GLN187 H.hehf.1 H.hehf.1 1
453 ALA188 None H.hehf.2 1
454 ASP189 H.hehf.3 H.hehf.3 1
455 TYR190 H.hehf.4 H.hehf.4 1
456 VAL191 H.hehf.5 H.hehf.5 1
457 PRO192 H.hehf.6 H.hehf.6 1
458 SER193 H.hehf.7 H.hehf.7 1

The consensus was kept, I am relabelling these:
H.hehf.2

G.hfs2
465 CYS200 G.hfs2.1 G.hfs2.1 1
466 ARG201 G.hfs2.2 G.hfs2.2 1
467 VAL202 G.hfs2.3 G.hfs2.3 1
468 LEU203 None G.hfs2.4 1
469 THR204 None G.hfs2.5 1
470 SER205 G.hfs2.6 G.hfs2.6 1
471 GLY206 G.hfs2.7 G.hfs2.7 1

The consensus was kept, I am relabelling these:
G.hfs2.4
G.hfs2.5

G.s4h3
515 SER250 G.s4h3.1 G.s4h3.1 1
516 SER251 G.s4h3.2 G.s4h3.2 1
517 SER252 G.s4h3.3 G.s4h3.3 1
518 TYR253 G.s4h3.4 G.s4h3.4 1
519 ASN254 None G.s4h3.5 1
520 VAL255 None G.s4h3.6 1
521 LEU256 None G.s4h3.7 1
522 ILE257 None G.s4h3.8 1
523 ARG258 None G.s4h3.9 1
524 GLU259 None G.s4h3.10 1
525 ASP260 None G.s4h3.11 1
526 ASN261 G.s4h3.5 G.s4h3.12 0
527 GLN262 None G.s4h3.13 0
528 THR263 G.s4h3.14 G.s4h3.14 0
529 ASN264 G.s4h3.15 G.s4h3.15 0

Consensus wasn't kept. Nothing done!

INFO: these are the CGN fragments mapped onto your topology
G.HN with 28 AAs THR9( 288)-VAL36 (315 ) (G.HN)
G.hns1 with 3 AAs TYR37( 316)-ALA39 (318 ) (G.hns1)
G.S1 with 7 AAs THR40( 319)-LEU46 (325 ) (G.S1)
G.s1h1 with 6 AAs GLY47( 326)-GLY52 (331 ) (G.s1h1)
G.H1 with 7 AAs LYS53( 332)-GLN59 (338 ) (G.H1)
H.HA with 26 AAs LYS88( 353)-LEU113 (378 ) (H.HA)
H.hahb with 9 AAs VAL114( 379)-PRO122 (387 ) (H.hahb)
H.HB with 14 AAs GLU123( 388)-ASN136 (401 ) (H.HB)
H.hbhc with 7 AAs VAL137( 402)-PRO143 (408 ) (H.hbhc)
H.HC with 12 AAs PRO144( 409)-GLU155 (420 ) (H.HC)
H.hchd with 1 AAs ASP156( 421)-ASP156 (421 ) (H.hchd)
H.HD with 12 AAs GLU157( 422)-GLU168 (433 ) (H.HD)
H.hdhe with 5 AAs TYR169( 434)-ASP173 (438 ) (H.hdhe)
H.HE with 13 AAs CYS174( 439)-LYS186 (451 ) (H.HE)
H.hehf with 7 AAs GLN187( 452)-SER193 (458 ) (H.hehf)
H.HF with 6 AAs ASP194( 459)-ARG199 (464 ) (H.HF)
G.hfs2 with 5 AAs CYS200( 465)-GLY206 (471 ) (G.hfs2)
G.S2 with 8 AAs ILE207( 472)-VAL214 (479 ) (G.S2)
G.s2s3 with 2 AAs ASP215( 480)-LYS216 (481 ) (G.s2s3)
G.S3 with 8 AAs VAL217( 482)-VAL224 (489 ) (G.S3)
G.s3h2 with 3 AAs GLY225( 490)-GLN227 (492 ) (G.s3h2)
G.H2 with 10 AAs ARG228( 493)-CYS237 (502 ) (G.H2)
G.h2s4 with 5 AAs PHE238( 503)-THR242 (507 ) (G.h2s4)
G.S4 with 7 AAs ALA243( 508)-ALA249 (514 ) (G.S4)
G.s4h3 with 7 AAs SER250( 515)-ASN264 (529 ) (G.s4h3)
G.H3 with 18 AAs ARG265( 530)-LEU282 (547 ) (G.H3)
G.h3s5 with 3 AAs ARG283( 548)-ILE285 (550 ) (G.h3s5)
G.S5 with 7 AAs SER286( 551)-ASN292 (557 ) (G.S5)
G.s5hg with 1 AAs LYS293( 558)-LYS293 (558 ) (G.s5hg)
G.HG with 17 AAs GLN294( 559)-ASP310 (575 ) (G.HG)
G.hgh4 with 10 AAs TYR311( 576)-THR320 (585 ) (G.hgh4)
G.H4 with 27 AAs PRO321( 586)-ARG347 (612 ) (G.H4)
G.h4s6 with 11 AAs ILE348( 613)-TYR358 (623 ) (G.h4s6)
G.S6 with 5 AAs CYS359( 624)-PHE363 (628 ) (G.S6)
G.s6h5 with 5 AAs THR364( 629)-ASP368 (633 ) (G.s6h5)
G.H5 with 26 AAs THR369( 634)-LEU394 (659 ) (G.H5)
Hit enter to continue!

Computing distances in the interface between fragments
0, 1
and
2, 3.
The interface is defined by the residues within 35.0 Angstrom of each other in the reference topology.
Computing interface...done!

From 106499 potential group_1-group_2 distances, the interface was reduced to only 8872 potential contacts.
If this number is still too high (i.e. the computation is too slow) consider using a smaller interface cutoff

100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 4/4 [00:00<00:00, 68.90it/s]
joblib.externals.loky.process_executor._RemoteTraceback: 00 frames. chunknr 0 frames 8001
"""
Traceback (most recent call last):
File "/home/lepoard/anaconda3/lib/python3.7/site-packages/joblib/externals/loky/process_executor.py", line 418, in _process_worker
r = call_item()
File "/home/lepoard/anaconda3/lib/python3.7/site-packages/joblib/externals/loky/process_executor.py", line 272, in call
return self.fn(*self.args, **self.kwargs)
File "/home/lepoard/anaconda3/lib/python3.7/site-packages/joblib/_parallel_backends.py", line 567, in call
return self.func(*args, **kwargs)
File "/home/lepoard/anaconda3/lib/python3.7/site-packages/joblib/parallel.py", line 225, in call
for func, args, kwargs in self.items]
File "/home/lepoard/anaconda3/lib/python3.7/site-packages/joblib/parallel.py", line 225, in
for func, args, kwargs in self.items]
File "/home/lepoard/anaconda3/lib/python3.7/site-packages/mdciao-0.1-py3.7.egg/mdciao/contacts.py", line 200, in per_xtc_ctc
jctcs, jidx_pairs, j_atompairs = compute_contacts(igeom, ctc_residxs_pairs, **mdcontacts_kwargs)
File "/home/lepoard/anaconda3/lib/python3.7/site-packages/mdciao-0.1-py3.7.egg/mdciao/contacts.py", line 2451, in compute_contacts
atom_distances = _md.compute_distances(traj, atom_pairs, periodic=periodic)
File "/home/lepoard/anaconda3/lib/python3.7/site-packages/mdtraj/geometry/distance.py", line 74, in compute_distances
out = np.empty((xyz.shape[0], pairs.shape[0]), dtype=np.float32)
MemoryError: Unable to allocate array with shape (8001, 660502) and data type float32
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/home/lepoard/anaconda3/bin/interface_ctc_analyzer.py", line 4, in
import('pkg_resources').run_script('mdciao==0.1', 'interface_ctc_analyzer.py')
File "/home/lepoard/anaconda3/lib/python3.7/site-packages/pkg_resources/init.py", line 666, in run_script
self.require(requires)[0].run_script(script_name, ns)
File "/home/lepoard/anaconda3/lib/python3.7/site-packages/pkg_resources/init.py", line 1469, in run_script
exec(script_code, namespace, namespace)
File "/home/lepoard/anaconda3/lib/python3.7/site-packages/mdciao-0.1-py3.7.egg/EGG-INFO/scripts/interface_ctc_analyzer.py", line 13, in
File "/home/lepoard/anaconda3/lib/python3.7/site-packages/mdciao-0.1-py3.7.egg/mdciao/command_line_tools.py", line 941, in interface
File "/home/lepoard/anaconda3/lib/python3.7/site-packages/mdciao-0.1-py3.7.egg/mdciao/contacts.py", line 158, in xtcs2ctcs
File "/home/lepoard/anaconda3/lib/python3.7/site-packages/joblib/parallel.py", line 934, in call
self.retrieve()
File "/home/lepoard/anaconda3/lib/python3.7/site-packages/joblib/parallel.py", line 833, in retrieve
self._output.extend(job.get(timeout=self.timeout))
File "/home/lepoard/anaconda3/lib/python3.7/site-packages/joblib/_parallel_backends.py", line 521, in wrap_future_result
return future.result(timeout=timeout)
File "/home/lepoard/anaconda3/lib/python3.7/concurrent/futures/_base.py", line 435, in result
return self.__get_result()
File "/home/lepoard/anaconda3/lib/python3.7/concurrent/futures/_base.py", line 384, in __get_result
raise self._exception
MemoryError: Unable to allocate array with shape (8001, 660502) and data type float32

[gph82]

@hbat1 this is not the image size issue you posted at the end of #8 ... can you check if decreasing the chunksize solves this?

[hbat1]

@gph82 Actually to solve this problem I did two things, I decreased cut off size from 35 to 25 because I did not need 35 and I decreased chunk size to the default which is 10000