[ERRO] bufio.Scanner: token too long

[Axze-rgb]

Hello, I have a tab file like this

head tmp.tab
dummy_TRINITY_DN39498_c0_g1_i2 len=256 path=[0:0-255]	ATGCCGCAAGAAAGACCTCAGAGGTTTATCTGAACAGTGTTTGATCCAGCAGGTGAAAATAGAAGTGTTCAGATAAACCTCTGTCCTTCAGCTATGTTGTTGTTCATTCCCTGGCTGAACACTGGGGATCGCCCTGTCTATGCATCCTCGCGGGCGCTTGCGGGGTCGCTGGGTGCGGTTCTTGGGGTGAGAGGATTGCGTGAGTGGCTGTTTTTTGTTGCCTCGTCACATGGGCAGCCGCACTCCATCTTCAGAG	
dummy__TRINITY_DN39438_c0_g1_i1 len=717 path=[0:0-274 2:275-607 4:608-716]	CGATCTGTTTTCCGGACACTGAGTGGGCCCCGGCTTATTAGGATTTTAAAGCACGACAATAGAGAGGGCAGAGACTATGTGCCAAACATACTGGAATCTAGTGGGGACAAGTTTTTACAAGTGATTCATGCATGTTGGTCGATATCTTGGTGGAGCTCCCCTGTCATCCTGACCGTGCTGTATCGACGAGGATATTTCAGCCACGAAGGAATGGTGTCCATTAGCAAGTTCCTGTTCTCTGTTGGGGTTGTCTATGTTGGTGCTTACTGTTTAAGAGGTCTGGGTCGGCTGACCAACCCTGACTACATGATGTTCATCAATGTGCTGGTTCAAGCCCTCCACTCTCCCAGCTTTCAGAACAGCAAGAGGCTGCTGCACCACTATGACTTTGAGTTCTGGGCTTCACCAGTAGACTTCCAGTGGAACGAAGGCTCAGAGAGCAAAGGGGAGCGGGTAGGACACAGCAAAGTTCAAGCACCTGTGGAATCCAACTCTTCTTCCGGGATCATGGGACTGCCCTTCAGGCTGCTCAGTTACTTTGCCATTCACACCTTTGGACGACGCATGGTCTATCCCGGGGCCACTTCTTTGATGCAGATGCTTGTAGGTTAGTAACCTTTTTCTCTCTTCTCTTTAAACTTTTTTTGTTAAATTTAGAAATCAATCATTTGTTATATATTTTGAGCATTGTCAGACCTGTTCAGTCTTAAAATTGTG	
dummy__TRINITY_DN39438_c0_g1_i2 len=1791 path=[0:0-274 2:275-607 3:608-1790]	CGATCTGTTTTCCGGACACTGAGTGGGCCCCGGCTTATTAGGATTTTAAAGCACGACAATAGAGAGGGCAGAGACTATGTGCCAAACATACTGGAATCTAGTGGGGACAAGTTTTTACAAGTGATTCATGCATGTTGGTCGATATCTTGGTGGAGCTCCCCTGTCATCCTGACCGTGCTGTATCGACGAGGATATTTCAGCCACGAAGGAATGGTGTCCATTAGCAAGTTCCTGTTCTCTGTTGGGGTTGTCTATGTTGGTGCTTACTGTTTAAGAGGTCTGGGTCGGCTGACCAACCCTGACTACATGATGTTCATCAATGTGCTGGTTCAAGCCCTCCACTCTCCCAGCTTTCAGAACAGCAAGAGGCTGCTGCACCACTATGACTTTGAGTTCTGGGCTTCACCAGTAGACTTCCAGTGGAACGAAGGCTCAGAGAGCAAAGGGGAGCGGGTAGGACACAGCAAAGTTCAAGCACCTGTGGAATCCAACTCTTCTTCCGGGATCATGGGACTGCCCTTCAGGCTGCTCAGTTACTTTGCCATTCACACCTTTGGACGACGCATGGTCTATCCCGGGGCCACTTCTTTGATGCAGATGCTTGTAGGAGGTATGCTGAACCAGGGTCGGGCAAATCTCCTTGAAGAGAAGAAAGGAATACGAGCCAAGCTGCTGACAGAAGACAACAGTGAGATTGACACCATCTTCGTGGACAGGCGCAGAGTTGGCAGCCGCTATGGAAACACACTGGTGGTGTGCTGTGAGGGCAATGCAGGGTTCTATGAGATGGGCTGCTCAGGGACCCCCCTGGAGTGTGGATACTCTGTGCTGGGCTGGAACCACCCAGGCTTTGCCGGCAGCTCTGGGGTACCATTCCCCGATTCAGAACAGAATGCCATTGATGCTGTGATGAAGTACGCCATCTACCGCCTGGGCTTCCAGCCACACACCATCGTTTTGTTTGCCTGGTCCATTGGTGGGTACAGTGCCTCCTGGGCAGCCATGAACTACCCCGATGTCAAATCTGTGATCCTGGATGCCACCTTTGATGATGTTGTTCCTCTGGCCCTGACCCAGATGCCTCAGTCATGGAGTGGCATTGTGACAATGGCGATGAGAAACCACATGAATCTGAACATTGCTGATCAAATTTTAAAATACCCAGGCCCCATCAAGCTCATACGCAGAGTCCGAGATGAAATCATTGCCACAGAGAAGACGGAGGGAATGAACTCAGTATTGTCATCCAACCGTGGCAATTTTCTGCTCATTCGGCTCCTGCAGCATCGGTACCCCAAAATTGTGGATCGTGAGACCACACCTCTCCTGGAAGACTTCCTTAGTGGTACCAGACAACACCAAGACAAAATGCTGATGGAGCACCATGTAGTGCTGGAGGAGTGTGAAGCCAAGCTGCGTAGCTACGCCCAGGACATCTCTTCCTCCTACCCCATGCTGATAGGGGACGGGGAAACACCAGCAGTCAGAAAGCAGCTGACATTATTCCTGGCCTACAAATACATGGAGGACTTTGATTCTACGCACTGCACTCCTCTACCGATGTTCTACCTGTCACCCCCCTGGTCACCAATGGATTGATCCACTTCCACCCAGCGTTTTTATTGAAAAGTTCTGTTGTCTCCAAAAATATTTGTGACTGCTTTTTTTGGTTTCTCATTGTTGCGGTTAATTTTGTTGCTGTTCAGTTTTTGTCAGTAGTGCTTCTTTTTTGTACTGTGTTTGTAGGCGTGACTCTTTGTGTAGTGTTCTGTGTGTATGTGTGTGTGCAAG	
dummy__TRINITY_DN39438_c0_g1_i3 len=1669 path=[1:0-152 2:153-485 3:486-1668]	ATCTCAACACACACACAGGCTATTCTGTTGTTTCTTCAACGTAAGCAGTCAAAAACAAAACAAAACAAAACAAACAAACAAGCAATACAACCATAACGAAACAGGGATAAGCAGCAGAAGAAGTGAGTGAGTGAAGAGCGCGGTATAATGGGCAGGTCTGGGTCGGCTGACCAACCCTGACTACATGATGTTCATCAATGTGCTGGTTCAAGCCCTCCACTCTCCCAGCTTTCAGAACAGCAAGAGGCTGCTGCACCACTATGACTTTGAGTTCTGGGCTTCACCAGTAGACTTCCAGTGGAACGAAGGCTCAGAGAGCAAAGGGGAGCGGGTAGGACACAGCAAAGTTCAAGCACCTGTGGAATCCAACTCTTCTTCCGGGATCATGGGACTGCCCTTCAGGCTGCTCAGTTACTTTGCCATTCACACCTTTGGACGACGCATGGTCTATCCCGGGGCCACTTCTTTGATGCAGATGCTTGTAGGAGGTATGCTGAACCAGGGTCGGGCAAATCTCCTTGAAGAGAAGAAAGGAATACGAGCCAAGCTGCTGACAGAAGACAACAGTGAGATTGACACCATCTTCGTGGACAGGCGCAGAGTTGGCAGCCGCTATGGAAACACACTGGTGGTGTGCTGTGAGGGCAATGCAGGGTTCTATGAGATGGGCTGCTCAGGGACCCCCCTGGAGTGTGGATACTCTGTGCTGGGCTGGAACCACCCAGGCTTTGCCGGCAGCTCTGGGGTACCATTCCCCGATTCAGAACAGAATGCCATTGATGCTGTGATGAAGTACGCCATCTACCGCCTGGGCTTCCAGCCACACACCATCGTTTTGTTTGCCTGGTCCATTGGTGGGTACAGTGCCTCCTGGGCAGCCATGAACTACCCCGATGTCAAATCTGTGATCCTGGATGCCACCTTTGATGATGTTGTTCCTCTGGCCCTGACCCAGATGCCTCAGTCATGGAGTGGCATTGTGACAATGGCGATGAGAAACCACATGAATCTGAACATTGCTGATCAAATTTTAAAATACCCAGGCCCCATCAAGCTCATACGCAGAGTCCGAGATGAAATCATTGCCACAGAGAAGACGGAGGGAATGAACTCAGTATTGTCATCCAACCGTGGCAATTTTCTGCTCATTCGGCTCCTGCAGCATCGGTACCCCAAAATTGTGGATCGTGAGACCACACCTCTCCTGGAAGACTTCCTTAGTGGTACCAGACAACACCAAGACAAAATGCTGATGGAGCACCATGTAGTGCTGGAGGAGTGTGAAGCCAAGCTGCGTAGCTACGCCCAGGACATCTCTTCCTCCTACCCCATGCTGATAGGGGACGGGGAAACACCAGCAGTCAGAAAGCAGCTGACATTATTCCTGGCCTACAAATACATGGAGGACTTTGATTCTACGCACTGCACTCCTCTACCGATGTTCTACCTGTCACCCCCCTGGTCACCAATGGATTGATCCACTTCCACCCAGCGTTTTTATTGAAAAGTTCTGTTGTCTCCAAAAATATTTGTGACTGCTTTTTTTGGTTTCTCATTGTTGCGGTTAATTTTGTTGCTGTTCAGTTTTTGTCAGTAGTGCTTCTTTTTTGTACTGTGTTTGTAGGCGTGACTCTTTGTGTAGTGTTCTGTGTGTATGTGTGTGTGCAAG	
dummy__TRINITY_DN39497_c0_g1_i2 len=204 path=[0:0-203]	CAGGGGGGTGGGGCCTTGTTCTCCAACCTGTGGTTCCCCAACAACCAACGGGCCACCTCCACCGCCATCTCTACCTTCATCGGCTATACCGGCAGTGCTCTGGCCTTTGTTCTGGGGCCCTCTCTTGTTCCCAGTCCTTCCGACGACGTCAACTCCAACCAGACGCTGCCTGACGGGAACGACACAAGCGGAGCCGACCTACAG	
dummy__TRINITY_DN39436_c0_g1_i1 len=269 path=[0:0-85 2:86-268]	TGTTTGTCTGTCACCTTGTCGGCAGGTCTGTCAGTCCTTGCCATTCAGGAGGCCAGAGGAAAAAAAAAAAGCAAAAAAAAAAGCTACAAAGACACCCTGAAAGCGTCACTGAAGGTCTTCAGCATCGATCACGACTCATGGGAGCAGGCAGCTTTGGACAGAGCGAAGTGGCGCTCAGCTGTCTACCATAGCGCGCAGAGCTGTGAATCCAACAGGACAGCGGCAGCTGAGCTGAGCAAACAGGCCAGGAAAGCCCCAGCCAGCGCACC	
dummy__TRINITY_DN39436_c0_g1_i2 len=456 path=[1:0-272 2:273-455]	CGTCAATATGCAATTGCTATAGCAGATTTCATGGTGTGCTGTGGAAGCACTTTTCGTGCATTTCCATATTGTCAACATAAAATTCATACAGTCCCAGACACCGAGGTCCTCATCAGTGCAGACCTGCCCAGTATCCGTACCATCCTTATGCAGTCATAGCTTCGTTGGGCAGGTCATGTAGTTCGCATGCCAGACCTCCGGCTCCCAAAGAAACTCTTATTCTGCAAACTCCAATACGGCAATTGCTCCCAAGAAGGCCAGAACAAAGCGCTTCAAAGACACCCTGAAAGCGTCACTGAAGGTCTTCAGCATCGATCACGACTCATGGGAGCAGGCAGCTTTGGACAGAGCGAAGTGGCGCTCAGCTGTCTACCATAGCGCGCAGAGCTGTGAATCCAACAGGACAGCGGCAGCTGAGCTGAGCAAACAGGCCAGGAAAGCCCCAGCCAGCGCACC	
dummy__TRINITY_DN39429_c0_g1_i1 len=1122 path=[0:0-927 2:928-1121]	CTGTTTCACACCTTCCCCAGCTTTTCCTTCAAGATGTTCTCGGACATGTCCAGCATGTCCGCAGCCACTGATCGGTCCCTACAGGGGGAAAATGCCTACCTGAGGGAGAACATAGAGAAAGAGCGCTATAGGAGAAAGCATTGTGAACAACAGATTCAGAGTCTGAATGCCAAGTTGCTGGAAATACAACAGCAGCTGGCTGTGGCTATCTCCACTGACAAGCGCAAGGACATCATGATTGACCAGCTGGATAAACAACTGGCCAAAGTAGTGGAAGGGTGGAAGAAGCGAGAAGCAGAAAAGGACGAGTACATGTCCTTGGTGATGAAGGAGAAGTCTCAGATTGAAGAAACGTTGCAGAAACAGCAGGCAATGATTGACAGCTTTGAGAAGGAGCTGGCCAACACAGTGGATGAGCTGAAGCAGGAGAAGGAGAACTCTGCAGAGCTGGTGGACCAGATGAAGGCCCAGCTGTTGTCAGCGGTGCATGACCAGCGTCACGCTGAGGAGATGTTGGCAGCAGAGAAGGAGCGGGTAACTCTGATGGAGAGAGAGTGGGATCAGCTAAAGGAGGCACGGGACTTGGCGGAAAAACGAGGACAGCAAGTCCAGGATCGACTTCATCAGGAGCAGGACAGCTGGTACCAGCGGGAACAGGAACTGGTGCACAAGATTGACCAGGTCAAGGAGGCCAACCTCAAGGTCATGCAGATGGAACGGGTGAAGCTAGAGGAGCAGATGAAGAAAGCTGAAGACCTGGAGGAGCAAGCCCACACAGCGAACACAGAGGTCAAGAGGCTGGAGATGGAAGTGGACTCAGCTGTGAGGGAGAAGGAGAGCCTCAAAGTGGAGATGGCTCTGATGGAGGCCAAATTTGAGAGTGCACAGCGCACCCTGGAGGCAGACCTGCGTGGTCAGATGGAGAAAGAGATCTCAGAGCAGGTGGGCGAGGTGCAGAGGCGAATGCGACAGGAGCAAGAGGAGCAGGGGGAGAAGCACCGCCAGTTGGTGTCTGAGCTCCACCAGCGGCACCAGCGTGACCTGGACATGCAGTTGGCCACCCTGCGCCAGGACCTGGGCCGGCGGGAGGATGACCTGAAGGAGCAGTTAGCAGAGATGGAG	
dummy__TRINITY_DN39429_c0_g1_i3 len=276 path=[1:0-81 2:82-275]	CCCCAGTTCCTTCATTTTGACTTCAGGTTGGCAGGTACCTGTAAACATTCTTCACTAACGCACTCCCACCCCTTTTTCTGTCAGATCTCAGAGCAGGTGGGCGAGGTGCAGAGGCGAATGCGACAGGAGCAAGAGGAGCAGGGGGAGAAGCACCGCCAGTTGGTGTCTGAGCTCCACCAGCGGCACCAGCGTGACCTGGACATGCAGTTGGCCACCCTGCGCCAGGACCTGGGCCGGCGGGAGGATGACCTGAAGGAGCAGTTAGCAGAGATGGAG	
dummy__TRINITY_DN39458_c0_g1_i1 len=301 path=[0:0-300]	AATGAATGTGTGTGTGTGTGAATGAATGTGTGTGAATGAAAATGTGCATGTGTGAATGAATGTGCATGTGTGTGAATGAATGTGTGTGTGTGTGAATGAATGTGTGTGAATGAAAATGTGCATGTGTGAATGAATGTGCATGTGTGTGAATGAATGTGTGTGTGTGTGAATAAATTGTGTATGTGTGTGAACGAATGTGTGTGTGAATAAGTTTGTGTGTGTGAATATGTGTGCATGCGCTCGTGTGAATGAATGTGTGTGTTTGTATGTATGTGTGAGAATGAATGTTTGTGTGTGTGTG	

Then I convet it back to fasta

seqkit tab2fx tmp.tab > without_conus_all.fa
[ERRO] bufio.Scanner: token too long

What is the "token" here ?
Note that the final fasta file is not empty but I don't know if the error is harmless or if some tab were not converted to fasta.

Thank you!

[Axze-rgb]

By the way I can send you the whole files if needed.

[shenwei356]

Yes, please send the compressed file.

[shenwei356]

I confirmed it's a bug for very long sequences like chr1 or human genome.

[shenwei356]

Hi @Axze-rgb , I fixed it.

• seqkit_windows_amd64.exe.tar.gz
• seqkit_linux_amd64.tar.gz
• seqkit_darwin_amd64.tar.gz

[Axze-rgb]

Waw thanks! you rock

[Axze-rgb]

Indeed it works perfecty.

Thanks for seqkit, it's a very useful piece of software.