Eaton&Ree CLI Tutorial Issue

[konradtaube]

I am trying to practice the CLI tutorial using the 2013 paper, but am having an issue pretty early on. Using the ipyrad command I am able to get steps 1 and 2 done just fine, but on step 3 the program does not work. Typing in ipyrad -p params-pedicularis.txt -s 3 -r results in the following error at the aligning clusters step:

ipyrad -p params-pedicularis.txt -s 3 -r
loading Assembly: pedicularis
from saved path: ~/analysis-ipyrad/pedicularis.json

---

ipyrad [v.0.9.64]
Interactive assembly and analysis of RAD-seq data

Parallel connection | DESKTOP-OJ5M50N: 12 cores

Step 3: Clustering/Mapping reads within samples
[####################] 100% 0:00:09 | dereplicating
[####################] 100% 0:04:03 | clustering/mapping
[####################] 100% 0:00:01 | building clusters
[####################] 100% 0:00:00 | chunking clusters
[####################] 100% 0:24:54 | aligning clusters

Encountered an Error.
Message: IndexError: string index out of range
Parallel connection closed.

Summary stats of Assembly pedicularis

                               state  reads_raw  reads_passed_filter

29154_superba_SRR1754715 2 696994 689996
30556_thamno_SRR1754720 2 1452316 1440314
30686_cyathophylla_SRR1754730 2 1253109 1206947
32082_przewalskii_SRR1754729 2 964244 955480
33413_thamno_SRR1754728 2 636625 626084
33588_przewalskii_SRR1754727 2 1002923 993873
35236_rex_SRR1754731 2 1803858 1787366
35855_rex_SRR1754726 2 1409843 1397068
38362_rex_SRR1754725 2 1391175 1379626
39618_rex_SRR1754723 2 822263 813990
40578_rex_SRR1754724 2 1707942 1695523
41478_cyathophylloides_SRR1754722 2 2199740 2185364
41954_cyathophylloides_SRR1754721 2 2199613 2176210

Full stats files

step 1: ./analysis-ipyrad/pedicularis_s1_demultiplex_stats.txt
step 2: ./analysis-ipyrad/pedicularis_edits/s2_rawedit_stats.txt
step 3: None
step 4: None
step 5: None
step 6: None
step 7: None

Any idea of what I can do to get this program running through step3?

[isaacovercast]

What version of python are you running?

[konradtaube]

Right now I have just been using my Windows terminal/Unix stuff. Is that a problem? I've gone through the beginner tutorial and advanced tutorial just fine

[isaacovercast]

What version of python? python -V If the intro and advanced tutorials worked it's weird the pedicularis doesn't work. Can you do this:

ipyrad -p params-pedicularis.txt -s 3 -d

[isaacovercast]

Are you sure you have plenty of free disk space? df -h

If you run out of disk space wacky things can happen.

[isaacovercast]

It works for me with both python 2.7 and 3.6

[konradtaube]

Python is 3.8.5

I am currently running the ipyrad -p params-pedicularis.txt -s 3 -d command, I will let you know what happens once it finishes! Do you think the version of python could be the issue?

[isaacovercast]

Did you check your disk space? df -h

[konradtaube]

I just did, I get the following:

df -h
Filesystem Size Used Avail Use% Mounted on
rootfs 466G 452G 14G 98% /
none 466G 452G 14G 98% /dev
none 466G 452G 14G 98% /run
none 466G 452G 14G 98% /run/lock
none 466G 452G 14G 98% /run/shm
none 466G 452G 14G 98% /run/user
tmpfs 466G 452G 14G 98% /sys/fs/cgroup
C:\ 466G 452G 14G 98% /mnt/c

Also, I ran the ipyrad -p params-pedicularis.txt -s 3 -d command and it also got stuck at 98/99% again, the time was just over an hour and a half on that particular step.

[isaacovercast]

Well, it looks like your disk is 98% full, so I _guarantee_ you are running out of disk space during step 3 and this is causing the crash. ipyrad assemblies can consume a TON of disk, so we recommend having so much free disk you don't even have to worry about it. You need to have at _least_ 10 times the size of the raw data in free disk space.

…

On Fri, Jan 29, 2021 at 6:27 PM Konrad Taube ***@***.***> wrote: I just did, I get the following: df -h Filesystem Size Used Avail Use% Mounted on rootfs 466G 452G 14G 98% / none 466G 452G 14G 98% /dev none 466G 452G 14G 98% /run none 466G 452G 14G 98% /run/lock none 466G 452G 14G 98% /run/shm none 466G 452G 14G 98% /run/user tmpfs 466G 452G 14G 98% /sys/fs/cgroup C:\ 466G 452G 14G 98% /mnt/c Also, I ran the ipyrad -p params-pedicularis.txt -s 3 -d command and it also got stuck at 98/99% again, the time was just over an hour and a half on that particular step. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#432 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABNSXP4TF7JOM7OV53OZJL3S4LVYLANCNFSM4WY7KG3A> .

[isaacovercast]

This also explains why you got through the simulated analyses. The simulated datasets are quite small. On Fri, Jan 29, 2021 at 6:32 PM isaac overcast <isaac.overcast@gmail.com> wrote:

…

Well, it looks like your disk is 98% full, so I _guarantee_ you are running out of disk space during step 3 and this is causing the crash. ipyrad assemblies can consume a TON of disk, so we recommend having so much free disk you don't even have to worry about it. You need to have at _least_ 10 times the size of the raw data in free disk space. On Fri, Jan 29, 2021 at 6:27 PM Konrad Taube ***@***.***> wrote: > I just did, I get the following: > > df -h > Filesystem Size Used Avail Use% Mounted on > rootfs 466G 452G 14G 98% / > none 466G 452G 14G 98% /dev > none 466G 452G 14G 98% /run > none 466G 452G 14G 98% /run/lock > none 466G 452G 14G 98% /run/shm > none 466G 452G 14G 98% /run/user > tmpfs 466G 452G 14G 98% /sys/fs/cgroup > C:\ 466G 452G 14G 98% /mnt/c > > Also, I ran the ipyrad -p params-pedicularis.txt -s 3 -d command and it > also got stuck at 98/99% again, the time was just over an hour and a half > on that particular step. > > — > You are receiving this because you commented. > Reply to this email directly, view it on GitHub > <#432 (comment)>, > or unsubscribe > <https://github.com/notifications/unsubscribe-auth/ABNSXP4TF7JOM7OV53OZJL3S4LVYLANCNFSM4WY7KG3A> > . >

[isaacovercast]

Free up some disk and try again. Or try on another computer. Good luck!