mergeBed ERROR: Requested column 10, but database file - only has fields 1 - 9.

[JiahuaQu]

Description of the bug

I ran the chipseq pipeline in HPC and used bowtie2 as the aligner.
After MACS2, the mergeBed step had error:
mergeBed ERROR: Requested column 10, but database file - only has fields 1 - 9.

Hope get your help. Thank you.
my.log
test.sh.o9822496.txt

Command used and terminal output

No response

Relevant files

No response

System information

nextflow # version: 22.10.6
HPC
SGE: Sun Grid Engine
Singularity
CentOS
nf-core-chipseq-2.0.0

[JoseEspinosa]

Are you updating the narrow_peak parameter using a custom.config with the -c flag. It is advisable not to do so and use the-params-file flag instead as explained here.

[JiahuaQu]

Are you updating the narrow_peak parameter using a custom.config with the -c flag. It is advisable not to do so and use the-params-file flag instead as explained here.

Thank you for the good suggestion.
I ran the program with the -params-file flag again, but failed with new errors.
Hope get your further help. Thank you.
fail_again.txt

[JoseEspinosa]

See this thread in the nf-core Slack for a possible fix. In short, the problem could be when using Singularity, that you probably need to add a custom bind path for Singularity to mount the directory.

[JiahuaQu]

Thank you very much, Jose and Phil. I got into slack.
I will try the binding suggestion in slack and give you any feedback afterwards.

[JoseEspinosa]

OK, thanks a lot for reporting 😄

[JiahuaQu]

A good news: the program ran with bowtie2 succeeded.
Then, I switched to try BWA, but this method still failed. I used exactly the same codes and config files except for changing bowtie2 and Bowtie2Index to bwa and BWAIndex.
Details in the below three log files.

1-error_sent_to_email.txt
2-my.log.txt
3-nohup_test.sh.o371946.txt

[JoseEspinosa]

I can reproduce the error by providing a directory with no index files, so could you do the following checks to try to see which could be the reason causing the issue. If you could paste here the output of the 3 it would be awesome.

1. Could you try to see if the index is correctly staged in the working dir by doing:

ls /wynton/group/rajkovic/jiahua/test/09-test_NextFlow-nf-core-chipseq/test/BWA/work/a2/47aba59b81052ee234b8e73e115091/

2. Could you try to see whether the command used to retrieve the index is not working correctly working in your system:

cd /wynton/group/rajkovic/jiahua/test/09-test_NextFlow-nf-core-chipseq/test/BWA/work/a2/47aba59b81052ee234b8e73e115091
find -L ./ -name "*.amb" | sed 's/.amb//'

3. Finally, could you check that you are really providing the necessary files in the folder you set to contain the bwa index:

ls /wynton/group/rajkovic/jiahua/software/NextFlow/genome/NCBI/GRCm38/references/Mus_musculus/NCBI/GRCm38/Sequence/BWAIndex

[JiahuaQu]

Thank you for the suggestion. The below are my output:

1. Could you try to see if the index is correctly staged in the working dir by doing:

ls /wynton/group/rajkovic/jiahua/test/09-test_NextFlow-nf-core-chipseq/test/BWA/work/a2/47aba59b81052ee234b8e73e115091/

2. Could you try to see whether the command used to retrieve the index is not working correctly working in your system:

cd /wynton/group/rajkovic/jiahua/test/09-test_NextFlow-nf-core-chipseq/test/BWA/work/a2/47aba59b81052ee234b8e73e115091
find -L ./ -name "*.amb" | sed 's/.amb//'

3. Finally, could you check that you are really providing the necessary files in the folder you set to contain the bwa index:

ls /wynton/group/rajkovic/jiahua/software/NextFlow/genome/NCBI/GRCm38/references/Mus_musculus/NCBI/GRCm38/Sequence/BWAIndex

I found there are additional two versions of BWAIndex inside the first level of BWAIndex folder, version0.5.x and version0.6.0, so I checked both of them too.

[JoseEspinosa]

OK, I think actually the problem is these/version* folders inside --bwa-index path. We will try to fix it. But in the meanwhile you could provide the indexes files in a different folder and removing the version005.x and the version0.6.0 of it. Please, let us know if this fixes the issue.

[JiahuaQu]

I did as you suggested and ran the pipeline successfully finally.
Thank you very much.

[JoseEspinosa]

OK, nice to hear that. I will close this issue and the other one then. Please, do not hesitate in asking us if you find any other issue.