New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

use on the MacOS #6

Open
suntaosimon opened this Issue Apr 21, 2017 · 14 comments

Comments

Projects
None yet
3 participants
@suntaosimon

suntaosimon commented Apr 21, 2017

Hi
I have tried to learn more about Vidjil and have a simple question, can I use this on a Mac OS?

It only shows,

The Vidjil algorithm has been successfully tested on the following platforms :

CentOS 6.3 amd64
CentOS 6.3 i386
Debian Squeeze 6.0
Debian Wheezy 7.0 amd64
Fedora 19
FreeBSD 9.2
Ubuntu 12.04 LTS amd64
Ubuntu 14.04 LTS amd64

I guess it can work on MacOS, but did not find the way to install it.

@mikael-s

This comment has been minimized.

Show comment
Hide comment
@mikael-s

mikael-s Apr 21, 2017

Member

Hi,

Yes it also works under MacOS. The installation instructions are provided in the doc/algo.org file, under the Installation section. Normally you just need to run:

make data
make germline
make

Please tell us if you encounter any trouble.
Also you're welcome to continue running your data on our public Vidjil server.

Member

mikael-s commented Apr 21, 2017

Hi,

Yes it also works under MacOS. The installation instructions are provided in the doc/algo.org file, under the Installation section. Normally you just need to run:

make data
make germline
make

Please tell us if you encounter any trouble.
Also you're welcome to continue running your data on our public Vidjil server.

@magiraud

This comment has been minimized.

Show comment
Hide comment
@magiraud

magiraud Apr 21, 2017

Contributor

Dear suntaosimon,

To complete the previous answer, Xcode should be installed.

Vidjil needs a C++11 compiler. When I was on OS.X 10.9, I had to use make CXXFLAGS='-std=c++11' LDFLAGS='-stdlib=libc++', as described in doc/algo.org. With OS X 10.11, it should work with the default compiler options.

Contributor

magiraud commented Apr 21, 2017

Dear suntaosimon,

To complete the previous answer, Xcode should be installed.

Vidjil needs a C++11 compiler. When I was on OS.X 10.9, I had to use make CXXFLAGS='-std=c++11' LDFLAGS='-stdlib=libc++', as described in doc/algo.org. With OS X 10.11, it should work with the default compiler options.

@suntaosimon

This comment has been minimized.

Show comment
Hide comment
@suntaosimon

suntaosimon Apr 21, 2017

Dear mikael-s and magiraud,

Thanks for your tips.

I have tried to type
make data
make germline
make
in the Terminal and it gives me "make: *** No rule to make target `data'. Stop."

I also use "wget http://bioinfo.lifl.fr/vidjil/vidjil-2017.03_x86_64 -O vidjil" and the file was downloaded. I don't know what's my next step. I also have installed Xcode.

I was using MIXCR for data analysis for now and the installation is easy by homebrew. I am not a bioinformatic person so I may asked some stupid questions.

suntaosimon commented Apr 21, 2017

Dear mikael-s and magiraud,

Thanks for your tips.

I have tried to type
make data
make germline
make
in the Terminal and it gives me "make: *** No rule to make target `data'. Stop."

I also use "wget http://bioinfo.lifl.fr/vidjil/vidjil-2017.03_x86_64 -O vidjil" and the file was downloaded. I don't know what's my next step. I also have installed Xcode.

I was using MIXCR for data analysis for now and the installation is easy by homebrew. I am not a bioinformatic person so I may asked some stupid questions.

@suntaosimon

This comment has been minimized.

Show comment
Hide comment
@suntaosimon

suntaosimon Apr 21, 2017

I think I have installed vidjil well by download vidjil-2017.03.tgz from http://bioinfo.lille.inria.fr/vidjil/
unzip the tgz file into the vidjil folder and used the following commands in the folder:
make data
make germline
make

I will need to analysis my incomplete IGH(dj) fq.gz data.

suntaosimon commented Apr 21, 2017

I think I have installed vidjil well by download vidjil-2017.03.tgz from http://bioinfo.lille.inria.fr/vidjil/
unzip the tgz file into the vidjil folder and used the following commands in the folder:
make data
make germline
make

I will need to analysis my incomplete IGH(dj) fq.gz data.

@mikael-s

This comment has been minimized.

Show comment
Hide comment
@mikael-s

mikael-s Apr 21, 2017

Member

Ok, good!

According to what you tell us, you'll have to launch vidjil in the following way:

./vidjil -r 1 -g germline/homo-sapiens.g:IGH,IGH+ file.fq.gz

You also have many other parameters, for instance depending on the number of clones you want to output, that are detailed in the documentation, as well as the output you'll get.
Among the output, you'll have a .vidjil file that can be viewed through the web application (the file can be loaded through the import/export menu).

Member

mikael-s commented Apr 21, 2017

Ok, good!

According to what you tell us, you'll have to launch vidjil in the following way:

./vidjil -r 1 -g germline/homo-sapiens.g:IGH,IGH+ file.fq.gz

You also have many other parameters, for instance depending on the number of clones you want to output, that are detailed in the documentation, as well as the output you'll get.
Among the output, you'll have a .vidjil file that can be viewed through the web application (the file can be loaded through the import/export menu).

@suntaosimon

This comment has been minimized.

Show comment
Hide comment
@suntaosimon

suntaosimon Apr 21, 2017

Thanks mikael, I used hiseq PE150 for the sequencing thus I have 2 reads fastq files.
BCR_R1.fq.gz
BCR_R2.fq.gz

I have tried the following command,
./vidjil -g germline/homo-sapiens.g:IGH -3 data1/BCR_R1.fq.gz data1/BCR_R2.fq.gz
but can not process 2 reads at same time. ([error] Wrong number of arguments.)
how can I process PE sequencing data?

I also tried
./vidjil -g germline/homo-sapiens.g:IGH -3 data1/BCR_R1.fq.gz
and
./vidjil -g germline/homo-sapiens.g:IGH -3 data1/BCR_R2.fq.gz

separately.

There are 4 files for each reads generated. As you mentioned .vidjil file that can be viewed through the web application (the file can be loaded through the import/export menu). Is this the only way to view the output?

Thanks.

suntaosimon commented Apr 21, 2017

Thanks mikael, I used hiseq PE150 for the sequencing thus I have 2 reads fastq files.
BCR_R1.fq.gz
BCR_R2.fq.gz

I have tried the following command,
./vidjil -g germline/homo-sapiens.g:IGH -3 data1/BCR_R1.fq.gz data1/BCR_R2.fq.gz
but can not process 2 reads at same time. ([error] Wrong number of arguments.)
how can I process PE sequencing data?

I also tried
./vidjil -g germline/homo-sapiens.g:IGH -3 data1/BCR_R1.fq.gz
and
./vidjil -g germline/homo-sapiens.g:IGH -3 data1/BCR_R2.fq.gz

separately.

There are 4 files for each reads generated. As you mentioned .vidjil file that can be viewed through the web application (the file can be loaded through the import/export menu). Is this the only way to view the output?

Thanks.

@mikael-s

This comment has been minimized.

Show comment
Hide comment
@mikael-s

mikael-s Apr 21, 2017

Member

Vidjil doesn't handle paired-end reads, they must be merged beforehand. On our public web server we use an external tool to do so (called PEAR). You can still merge the reads on our server and then download the merged reads, if you want to launch Vidjil yourself… but that's not very convenient.

Member

mikael-s commented Apr 21, 2017

Vidjil doesn't handle paired-end reads, they must be merged beforehand. On our public web server we use an external tool to do so (called PEAR). You can still merge the reads on our server and then download the merged reads, if you want to launch Vidjil yourself… but that's not very convenient.

@suntaosimon

This comment has been minimized.

Show comment
Hide comment
@suntaosimon

suntaosimon Apr 21, 2017

So, a simple protocol for me is,

  1. use PEAR or http://app.vidjil.org/ to merge PE fastq files into 1 fastq file
  2. use vidijil command, such as, ./vidjil -r 1 -g germline/homo-sapiens.g:IGH,IGH+ file.fq.gz to generate .vidijil files.
  3. import .vidjil files to http://app.vidjil.org/ and analysis from there.

Is there any way to export a human readable file (txt or csv) from local? such as mixcr "export" command.

Thanks.

suntaosimon commented Apr 21, 2017

So, a simple protocol for me is,

  1. use PEAR or http://app.vidjil.org/ to merge PE fastq files into 1 fastq file
  2. use vidijil command, such as, ./vidjil -r 1 -g germline/homo-sapiens.g:IGH,IGH+ file.fq.gz to generate .vidijil files.
  3. import .vidjil files to http://app.vidjil.org/ and analysis from there.

Is there any way to export a human readable file (txt or csv) from local? such as mixcr "export" command.

Thanks.

@suntaosimon

This comment has been minimized.

Show comment
Hide comment
@suntaosimon

suntaosimon Apr 24, 2017

image
Looks like there are only 200 clones can be extracted. Is there a way to see all the clones? We are looking for the MRD data.
Thanks.

suntaosimon commented Apr 24, 2017

image
Looks like there are only 200 clones can be extracted. Is there a way to see all the clones? We are looking for the MRD data.
Thanks.

@mikael-s

This comment has been minimized.

Show comment
Hide comment
@mikael-s

mikael-s Apr 25, 2017

Member

Is there any way to export a human readable file (txt or csv) from local? such as mixcr "export" command.

The program output some information when it is launched (as the number of sequence analysed, how many are not analysed and why). It also outputs the top clones.

Looks like there are only 200 clones can be extracted. Is there a way to see all the clones? We are looking for the MRD data.

It depends on what are you looking for.

  • Are you looking for a clonotype that was identified at diagnosis?
  • Otherwise, do you have the DNA sequence you are looking for?
  • Do you want to do a full repertoire analysis?

Vidjil can process as many clonotes as one wants. However the browser cannot display too many clonotypes as it would become quickly unusable.

Member

mikael-s commented Apr 25, 2017

Is there any way to export a human readable file (txt or csv) from local? such as mixcr "export" command.

The program output some information when it is launched (as the number of sequence analysed, how many are not analysed and why). It also outputs the top clones.

Looks like there are only 200 clones can be extracted. Is there a way to see all the clones? We are looking for the MRD data.

It depends on what are you looking for.

  • Are you looking for a clonotype that was identified at diagnosis?
  • Otherwise, do you have the DNA sequence you are looking for?
  • Do you want to do a full repertoire analysis?

Vidjil can process as many clonotes as one wants. However the browser cannot display too many clonotypes as it would become quickly unusable.

@suntaosimon

This comment has been minimized.

Show comment
Hide comment
@suntaosimon

suntaosimon Apr 25, 2017

The program

Refers to the online Vidjil or the local program?

It depends on what are you looking for.

Yes, we are looking for a clonotype that was identified at diagnosis (Pre-treatment). And also, we would like to do a full repertoire analysis(both TCR and BCR data). It's true that the browser cannot display too many clonotypes but this there a way to export to csv file with local program?

Thanks.

suntaosimon commented Apr 25, 2017

The program

Refers to the online Vidjil or the local program?

It depends on what are you looking for.

Yes, we are looking for a clonotype that was identified at diagnosis (Pre-treatment). And also, we would like to do a full repertoire analysis(both TCR and BCR data). It's true that the browser cannot display too many clonotypes but this there a way to export to csv file with local program?

Thanks.

@suntaosimon

This comment has been minimized.

Show comment
Hide comment
@suntaosimon

suntaosimon Apr 28, 2017

How to export a csv file with all clones starting from the first read? The web only allows to export 200 as maximum.
image

Thanks!

suntaosimon commented Apr 28, 2017

How to export a csv file with all clones starting from the first read? The web only allows to export 200 as maximum.
image

Thanks!

@mikael-s

This comment has been minimized.

Show comment
Hide comment
@mikael-s

mikael-s May 5, 2017

Member

Yes, we are looking for a clonotype that was identified at diagnosis (Pre-treatment). And also, we would like to do a full repertoire analysis(both TCR and BCR data). It's true that the browser cannot display too many clonotypes but this there a way to export to csv file with local program?

There are two aspects.

  1. For a clonotype identified at diagnosis: You have to launch Vidjil on each sample separately. Thus you will have one .vidjil file per sample. Then what you want to do is to produce one vidjil file from this set. To do so you should use the tools/fuse.py script. The fuse.py script will keep track of the top 50 (by default) clones in each sample. Thus it will allow to follow the main clones at diagnosis in the following samples, even if they are at very low concentration.
  2. For a full repertoire analysis: It also depends in what you're interested to see in the full repertoire. Assuming you're interested in the V(D)J recombinations of all the clones, you can launch Vidjil with the -A option. However, this is not the intended use for Vidjil, and this may be very slow.

The “export csv” feature in the web client only exports the visible clones (as stated), so it cannot export all of them.
If you're interested in files describing the clones, please pay attention to the output files produced by Vidjil. Then you can parse those files and do whatever you'd like with them. You may also be interested in the convert tool in vdjtools.

Member

mikael-s commented May 5, 2017

Yes, we are looking for a clonotype that was identified at diagnosis (Pre-treatment). And also, we would like to do a full repertoire analysis(both TCR and BCR data). It's true that the browser cannot display too many clonotypes but this there a way to export to csv file with local program?

There are two aspects.

  1. For a clonotype identified at diagnosis: You have to launch Vidjil on each sample separately. Thus you will have one .vidjil file per sample. Then what you want to do is to produce one vidjil file from this set. To do so you should use the tools/fuse.py script. The fuse.py script will keep track of the top 50 (by default) clones in each sample. Thus it will allow to follow the main clones at diagnosis in the following samples, even if they are at very low concentration.
  2. For a full repertoire analysis: It also depends in what you're interested to see in the full repertoire. Assuming you're interested in the V(D)J recombinations of all the clones, you can launch Vidjil with the -A option. However, this is not the intended use for Vidjil, and this may be very slow.

The “export csv” feature in the web client only exports the visible clones (as stated), so it cannot export all of them.
If you're interested in files describing the clones, please pay attention to the output files produced by Vidjil. Then you can parse those files and do whatever you'd like with them. You may also be interested in the convert tool in vdjtools.

@suntaosimon

This comment has been minimized.

Show comment
Hide comment
@suntaosimon

suntaosimon May 10, 2017

Thanks for your time and help.
I have spent some time to try to follow your advice but I still have some questions.

  1. I have tried to use tools/fuse.py with the command,
    python tools/fuse.py --output mrd.vidjil --top 100 out/BCR01-01.fq.vidjil out/BCR23-01.fq.vidjil
    it gives me an error code :
    File "", line 1
    python tools/fuse.py --output mrd.vidjil --top 100 out/BCR01-01.fq.vidjil out/BCR23-01.fq.vidjil
    ^
    SyntaxError: invalid syntax
    BUT besides the error code, the more important concern to me is, fuse.py is a merge process of 2 vidjil files with top 100 clones, maybe we can make it to top 10,000 or even more(what if the target clone we are tracking is at very low concentration, maybe, let's say, number 10,000). After I got the mrd.vidjil file and I feed it to the web client, I still can only see the first 200, right? If the target clone is at number 30,000 in the after-treatment sample, I still can not see it in the web app. I didn't get the merged vidjil file due the above error, so I was just thinking in this way.

  2. For a full repertoire analysis, you have introduced me an -A option and this will process all the clones. I guess I could use the command like,
    ./vidjil -A -r 1 -g germline/homo-sapiens.g:IGH,IGH+,IGK data1/BCR01-01.fq.gz
    I still could not use this vidjil output with the web client, right? Because it only can analysis top 200. So I have to use vdjtools by converting the vidjil file into vdjtools format and analysis from there. Because VDJtools will only use top clonotypes which have V/D/J detalization in the output, some clones lacking of V/D/J detalization will be dropped. So if our target clone is in this range, we can not see it, right?
    VDJtools supports parsing output Json files, what's Json file produced by vidjil, I only have .fa, .vidjil and a .windows.fa files.
    I have tried
    Convert \ -S Vidjil BCR01-01.fq.vidjil output
    the error messeges are,

Executing com.antigenomics.vdjtools.misc.Convert -S Vidjil BCR01-01.fq.vidjil output
[Wed May 10 00:33:25 CDT 2017 Convert] Reading sample(s)
[Wed May 10 00:33:25 CDT 2017 Convert] 1 sample(s) loaded
[Wed May 10 00:33:25 CDT 2017 SampleStreamConnection] Loading sample BCR01-01.fq
[ERROR] java.lang.NullPointerException, see _vdjtools_error.log for details

Thank you so much!

suntaosimon commented May 10, 2017

Thanks for your time and help.
I have spent some time to try to follow your advice but I still have some questions.

  1. I have tried to use tools/fuse.py with the command,
    python tools/fuse.py --output mrd.vidjil --top 100 out/BCR01-01.fq.vidjil out/BCR23-01.fq.vidjil
    it gives me an error code :
    File "", line 1
    python tools/fuse.py --output mrd.vidjil --top 100 out/BCR01-01.fq.vidjil out/BCR23-01.fq.vidjil
    ^
    SyntaxError: invalid syntax
    BUT besides the error code, the more important concern to me is, fuse.py is a merge process of 2 vidjil files with top 100 clones, maybe we can make it to top 10,000 or even more(what if the target clone we are tracking is at very low concentration, maybe, let's say, number 10,000). After I got the mrd.vidjil file and I feed it to the web client, I still can only see the first 200, right? If the target clone is at number 30,000 in the after-treatment sample, I still can not see it in the web app. I didn't get the merged vidjil file due the above error, so I was just thinking in this way.

  2. For a full repertoire analysis, you have introduced me an -A option and this will process all the clones. I guess I could use the command like,
    ./vidjil -A -r 1 -g germline/homo-sapiens.g:IGH,IGH+,IGK data1/BCR01-01.fq.gz
    I still could not use this vidjil output with the web client, right? Because it only can analysis top 200. So I have to use vdjtools by converting the vidjil file into vdjtools format and analysis from there. Because VDJtools will only use top clonotypes which have V/D/J detalization in the output, some clones lacking of V/D/J detalization will be dropped. So if our target clone is in this range, we can not see it, right?
    VDJtools supports parsing output Json files, what's Json file produced by vidjil, I only have .fa, .vidjil and a .windows.fa files.
    I have tried
    Convert \ -S Vidjil BCR01-01.fq.vidjil output
    the error messeges are,

Executing com.antigenomics.vdjtools.misc.Convert -S Vidjil BCR01-01.fq.vidjil output
[Wed May 10 00:33:25 CDT 2017 Convert] Reading sample(s)
[Wed May 10 00:33:25 CDT 2017 Convert] 1 sample(s) loaded
[Wed May 10 00:33:25 CDT 2017 SampleStreamConnection] Loading sample BCR01-01.fq
[ERROR] java.lang.NullPointerException, see _vdjtools_error.log for details

Thank you so much!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment