New JSON format on MultiQC 1.20

[fgvieira]

Does TidyMultiqc supports the new json format on MultiQC version 1.20 (MultiQC/MegaQC#519)?

[multimeric]

No, but thanks for the reminder. I'll take a look.

[multimeric]

So it seems that the format is fundamentally the same, the only difference is that it now uses Plotly, so I guess that has resulted in a change in the plot data format. For example, .report_plot_data.qualimap_coverage_histogram.datasets is different between the two formats.

Likely I will just have to add some more plot parsers and possibly update the vignette. However, the default functionality works as is, especially if you don't want to extract plot data.

[multimeric]

Hi @fgvieira, can you please test if my branch works for your use case? You can test it using remotes::install_github("multimeric/TidyMultiqc", ref="multiqc_1.2").

[fgvieira]

Thanks for the super fast reply!

Parsing the general and raw, it seems to work fine:

df <- load_multiqc("multiqc_data.json",
  sections=c("general", "raw"),
  find_metadata = function(sample, parsed) {
    parsed[c(
      "config_creation_date",
      "config_version",
      "config_output_dir"
    )]
  }
)

But, when parsing plot:

> df <- load_multiqc("multiqc_data.json",
  sections=c("general", "raw", "plot"), plots=list_plots("multiqc_data.json")$id,
  find_metadata = function(sample, parsed) {
    parsed[c(
      "config_creation_date",
      "config_version",
      "config_output_dir"
    )]
  }
)

I get an error that seems to be related to modules that were run multiple times.

Example data:
multiqc_data.json.zip

[multimeric]

I really don't recommend that you try to grab all the data like this. The general data is generally more sensible than raw, so only use the latter if absolutely essential. In terms of plots, it's not trivial to implement parsers for each plot type, so I've only done so for a small subset. If there is a specific plot that you think you need for your analysis then feel free to open an issue about it. However, requesting all plot data doesn't really make sense to me. If you want all the data, or you want to answer a very specific question about the data, then I would suggest loading the JSON file yourself in R.