Allow a file with a list of files as input #201
Conversation
|
Aha, yes I can see how that could be a problem. I can't say that I ever expected anyone to specify every single log file :) This should be fairly easy to implement and your changes look good. I might suggest a couple of minor changes in strategy in inline comments though if that's ok 😉 Note that I'm probably going to rewrite this code fairly soon. You just prompted me to write down my intentions, so now described in #202. Shouldn't affect this change though. Phil |
|
Overarching code suggestions in one place here first:
Will follow these up with inline comments. |
| @@ -27,6 +27,7 @@ | |||
| creation_date = datetime.now().strftime("%Y-%m-%d, %H:%m") | |||
| working_dir = os.getcwd() | |||
| analysis_dir = [os.getcwd()] | |||
| list_files = None | |||
|
Oh, and please also add the parameter to the example config file and add a bit to the docs about it if that's ok. Thanks! Phil |
|
Actually, perhaps my suggestion about using |
|
I followed your first comment to convert file_list in a flag. I liked more, but is not perfect either of them. Let me know what you think now. Maybe we can leave it as a flag, and change it later depending of the user suggestions ... Thanks!!!! |
| if is_file_list: | ||
| if len(analysis_dir) > 1: | ||
| raise ValueError("If --is-file-list is giving, analysis_dir should have only one plain text file.") | ||
| if os.path.isdir(analysis_dir[0]): |
There was a problem hiding this comment.
You can probably skip this bit - the open command will throw an exception if it's a directory anyway I think.
|
Looks good! Couple of minor comments but then will merge. We also need to add a demo file list to MultiQC_TestData and update |
|
Ok, added a test file so you should now be able to add a test command to the travis config: |
| - multiqc --is-file-list data/special_cases/file_list.txt |
There was a problem hiding this comment.
Sorry, you'll need -f here to get it to overwrite the existing report.
Add -f to travis.
|
Sorry, just saw this. Looks good! |
|
Thanks! |
|
Hi @lpantano, I just changed Phil |
|
Sorry, shouldn't push changes when I'm tired! New commit with changes actually working now. |
|
thanks! |
…-3.1-and-lt-5.0 Update flake8-debugger requirement from ~=3.1 to >=3.1,<5.0
Hi!
this is a suggestion PR, since probably you have a better idea how to integrate it.
But we had some problems with a person with so many arguments (files) in the command line that it cannot run. (bcbio/bcbio-nextgen#1380)
In bcbio, we are moving to CWL, what has many advantages, but one of the inconvenient is that we cannot use commands assuming that we are in the same machine or in a standard file system. So we can not use
multiqc qc/*etc. Instead we are passing the specific path.So, @chapmanb thought that maybe it would be good to have an argument to allow a file that contains a list of files to be used by multiqc.
As I said, this is a suggestion, because I like to offer some kind of solution instead of just asking, but feel free to adapt however you think is better.
thanks!
PS: another problem it is if 700 samples make any sense inside multiqc, but I think we can change things to allow a proper report in the future.