Skip to content

Spectrum Plots#16

Merged
axelwalter merged 31 commits intoOpenMS:mainfrom
axelwalter:spectrum-plots
Jul 31, 2024
Merged

Spectrum Plots#16
axelwalter merged 31 commits intoOpenMS:mainfrom
axelwalter:spectrum-plots

Conversation

@axelwalter
Copy link
Collaborator

Added the full functionality for spectrum plots.

Due to time constraints there are no example notebooks yet (will be in a separate PR). However, I included a temporary streamlit app file showcasing everything (app-devel-spectrum.py).

Some questions / discussion points remain:

  • new functionality to plot heatmaps in scatter (needed in feature heatmap, ion mobility spectrum, peak map...)
    • any comments on the implementation welcome, since I am not sure how this integrates with user passing backend specific parameters
  • changed parameter names x to mz and y to intensity for spectra because of ion mobility plots, should it stay x and y for consistency with others?
  • need help with Bokeh tooltips (works with ion mobility, but not with vline spectra)
  • matplotlib grid lines on top of scatter in ion mobility plot
  • how to update global settings (e.g. axis labels) setting self.xlabel = "new label" in plot methods does not transfer to _create_figure method: want to automatically set default axis labels if ion mobility is set

@axelwalter axelwalter requested review from jcharkow and singjc and removed request for singjc July 18, 2024 12:52
@singjc
Copy link
Collaborator

singjc commented Jul 19, 2024

new functionality to plot heatmaps in scatter (needed in feature heatmap, ion mobility spectrum, peak map...)
any comments on the implementation welcome, since I am not sure how this integrates with user passing backend specific parameters

I'm not too sure I understand what is needed? Do we need to implement a different kind of heatmap plot?

changed parameter names x to mz and y to intensity for spectra because of ion mobility plots, should it stay x and y for consistency with others?

I personally would prefer to leave it as x and y for consistency with the other kinds of plots, when using the plot method. Otherwise, it could get confusing knowing which to use when wanting a different kind of plot?

I was looking into including a higher-level plot method for each specific kind, i.e. df.plot_spectrum, df.plot_chromatogram, etc. With these we could then do df.plot_spectrum(mz = "mz", intensity = "int"), df.plot_chromatogram(rt = "rt", intensity = "int"). Pandas can be extended with additional accessors, which is how we can attach these higher-level plot methods to the dataframe object. Still currently looking into this to see how easy/feasible it is.

See: https://pandas.pydata.org/pandas-docs/stable/development/extending.html

need help with Bokeh tooltips (works with ion mobility, but not with vline spectra)

I can try look into this. Did you use the new app-devel-spectrum.py to test for this?

matplotlib grid lines on top of scatter in ion mobility plot

Do we need to draw the grid lines before drawing the scatter points?

how to update global settings (e.g. axis labels) setting self.xlabel = "new label" in plot methods does not transfer to _create_figure method: want to automatically set default axis labels if ion mobility is set

The base plot config is used to update all the defaults if specific parameters (i.e. xlabel) is not passed directly to the plot method.

So if df.plot("rt", "int", "chromatogram", xlabel = "RT"), then self.xlabel should be "RT", otherwise if df.plot("rt", "int", "chromatogram") then self.xlabel should be "X-axis", because it gets updated by the config.

What we could do, is add a mapping for different axis labels that correspond to their kind of plot, and then we can pass kind to the _BasePlotConfig to configure the default values for axis-labels and other params that of plot specific.

Copy link
Collaborator

@singjc singjc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I briefly skimmed, but everything looks fine to me. I was wondering about two things though:

  1. When a Spectrum plot is requested, and ion_mobility is set, then it generates an ion mobility vs m/z PeakMap. I think this can already be done by using the feature_heatmap (which we can rename to PeakMap) kind of plot instead? I just think it might be confusing, if you're asking for a spectrum plot, but provide an ion mobility column to generate a PeakMap plot?

  2. For the scatter plot marker cycler, if we use by to plot by a grouped df, I'm not sure how this will look for a RT vs IM peak map, but I think it might look weird with all the different markers? The RT vs IM peak map is more dense then a MZ vs IM peak map, so all the different markers might make the RT vs IM peak map hard to visualize.

There is a README in the test/test_data that is still in the pyopenms_viz folder, I don't think it got moved with the rest of test/test_data to the main project folder for some reason. Can you add some info in that README about the two new test tsv files, just so we know how they were generated / where they come from.

@jcharkow may have more suggestions / comments.

Comment on lines +534 to +535
mz: str,
intensity: str,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would prefer to keep these as x and y

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, that's what I would prefer too. Just did that for the ion mobility option to make things clear. Acutally I really like the suggestion to rename the FeatureHeatmapPlot to a generic PeakMap which can plot ion mobility spectra, MS experiments, feature heatmaps etc. Then we could keep it simply with x and y and the PeakMap would have an optional "intensity".

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes I would keep "x" and "y" too, For 3D maps I would label "z" instead of intensity as well

if self.ion_mobility is None:
self.plot(spectrum, reference_spectrum, mz, intensity, **kwargs)
else:
self.plot_ion_mobility(spectrum, mz, ion_mobility, intensity, **kwargs)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Couldn't this be done with the FeatureHeatmap (which we can rename to PeakMap) Plot? i.e. df.plot(kind='feature_heatmap', x='mz', y='im')?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The reason for implementing it in scatter was to plot universal peak maps with scatter. But renaming the FeatureHeatMap to PeakMap would be very nice and clear.

def plot(self, x, y, **kwargs):
def plot(self, spectrum, reference_spectrum, x, y, **kwargs):
"""Standard spectrum plot with m/z on x-axis, intensity on y-axis and optional mirror spectrum."""
kwargs.pop("fig", None) # remove figure from **kwargs if exists
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why does this occur?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Passing the spectrum and reference_spectrum? I modify them with the prepare_data method (relative intensities etc). before calling either the normal spectrum plot method or the ion mobility spectrum plot.

@axelwalter
Copy link
Collaborator Author

I briefly skimmed, but everything looks fine to me. I was wondering about two things though:

1. When a Spectrum plot is requested, and ion_mobility is set, then it generates an ion mobility vs m/z PeakMap. I think this can already be done by using the feature_heatmap (which we can rename to PeakMap) kind of plot instead? I just think it might be confusing, if you're asking for a spectrum plot, but provide an ion mobility column to generate a PeakMap plot?

Yes, best solution!

2. For the scatter plot marker cycler, if we use `by` to plot by a grouped df, I'm not sure how this will look for a RT vs IM peak map, but I think it might look weird  with all the different markers? The RT vs IM peak map is more dense then a MZ vs IM peak map, so all the different markers might make the RT vs IM peak map hard to visualize.

Hm it is optional and in such a case the user doesn't need to use "by"? It looks nice for scatter plots which are not too crowded. I get your concern but would leave that to the user to chose the right settings which look good.

@axelwalter
Copy link
Collaborator Author

new functionality to plot heatmaps in scatter (needed in feature heatmap, ion mobility spectrum, peak map...)
any comments on the implementation welcome, since I am not sure how this integrates with user passing backend specific parameters

I'm not too sure I understand what is needed? Do we need to implement a different kind of heatmap plot?

The question was how can the user modify the markers in the current setup when we create a marker_dict for the heat map it is kind of pre-defined. One way would be to update the marker_dict with values from a custom passed marker_dict. But this is probably not very important in real use cases.

changed parameter names x to mz and y to intensity for spectra because of ion mobility plots, should it stay x and y for consistency with others?

I personally would prefer to leave it as x and y for consistency with the other kinds of plots, when using the plot method. Otherwise, it could get confusing knowing which to use when wanting a different kind of plot?

True, and if we plot ion mobility spectra as peak maps that will be perfectly fine.

I was looking into including a higher-level plot method for each specific kind, i.e. df.plot_spectrum, df.plot_chromatogram, etc. With these we could then do df.plot_spectrum(mz = "mz", intensity = "int"), df.plot_chromatogram(rt = "rt", intensity = "int"). Pandas can be extended with additional accessors, which is how we can attach these higher-level plot methods to the dataframe object. Still currently looking into this to see how easy/feasible it is.

See: https://pandas.pydata.org/pandas-docs/stable/development/extending.html

This would be nice to have but the current method by chosing the kind of plot seems simple enough. I would set this as a potential future feature if on time constraints right now.

need help with Bokeh tooltips (works with ion mobility, but not with vline spectra)

I can try look into this. Did you use the new app-devel-spectrum.py to test for this?

Thanks! Yes, tested everything with the app-devel-spectrum.py. I changed the method to get tooltips a bit where you pass a dict of tooltip item names as appering in the tooltip as keys and dataframe columns as values. This works in Plotly entirely and in the Bokeh ion mobility plot, but not in the Bokeh spectrum plot.

matplotlib grid lines on top of scatter in ion mobility plot

Do we need to draw the grid lines before drawing the scatter points?

Would need to remove the order, should work yes. First update plot aes and then plot.

how to update global settings (e.g. axis labels) setting self.xlabel = "new label" in plot methods does not transfer to _create_figure method: want to automatically set default axis labels if ion mobility is set

The base plot config is used to update all the defaults if specific parameters (i.e. xlabel) is not passed directly to the plot method.

So if df.plot("rt", "int", "chromatogram", xlabel = "RT"), then self.xlabel should be "RT", otherwise if df.plot("rt", "int", "chromatogram") then self.xlabel should be "X-axis", because it gets updated by the config.

What we could do, is add a mapping for different axis labels that correspond to their kind of plot, and then we can pass kind to the _BasePlotConfig to configure the default values for axis-labels and other params that of plot specific.

If we have a generic PeakMap Plot which will be used to plot feature heatmaps, ion mobility spectra and experiments we can set sane defaults right away and don't need to change that (like now with spectrum and ion mobility spec).

@axelwalter
Copy link
Collaborator Author

@singjc thanks for the review and feedback! If @jcharkow agrees with changing FeatureHeatmap to PeakMap which will be used to plot the different kinds of peak maps (feature heatmaps, ion mobility spectra, MS experiments) I will implement that next week and push changes.

@jcharkow
Copy link
Collaborator

@singjc thanks for the review and feedback! If @jcharkow agrees with changing FeatureHeatmap to PeakMap which will be used to plot the different kinds of peak maps (feature heatmaps, ion mobility spectra, MS experiments) I will implement that next week and push changes.

Yes I agree with changing FeatureHeatMap to PeakMap. Will review the rest of the code shortly

Copy link
Collaborator

@jcharkow jcharkow left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have not tested it but looks good. Just a few minor suggestions. I like the idea of combining FeatureHeatMap into a PeakMap object and am excited to see those changes.

@axelwalter
Copy link
Collaborator Author

Thanks for the suggestions @singjc and @jcharkow ! Implemented the changes and added example notebooks, where the plotting backend can be selected in the first cell. The example streamlit app could be implemented in the OpenMS web app template, and we stick with notebooks here, what do you think?

@singjc
Copy link
Collaborator

singjc commented Jul 23, 2024

Thanks for the suggestions @singjc and @jcharkow ! Implemented the changes and added example notebooks, where the plotting backend can be selected in the first cell. The example streamlit app could be implemented in the OpenMS web app template, and we stick with notebooks here, what do you think?

Great, thanks for making the changes! I think that sounds good and makes sense 👍. We can probably include a link to the OpenMS web app template repo in the README to redirect/showcase using the plotting framework in a web app?

@axelwalter
Copy link
Collaborator Author

Great, thanks for making the changes! I think that sounds good and makes sense 👍. We can probably include a link to the OpenMS web app template repo in the README to redirect/showcase using the plotting framework in a web app?

Sure sounds great! Will add a pyopenms-viz page to the template app and link here in the README once this repo is public / released.

@axelwalter
Copy link
Collaborator Author

One thing missing here which I will add eventually is the 3D peak map with matplotlib.

@singjc
Copy link
Collaborator

singjc commented Jul 23, 2024

One thing missing here which I will add eventually is the 3D peak map with matplotlib.

Oh right, I think you did have code to do this in the MSExperimentPlotter in the initial version? Do you think we could re-use some of that code? Do we need to add another base class for 3D peak maps?

@axelwalter
Copy link
Collaborator Author

Oh right, I think you did have code to do this in the MSExperimentPlotter in the initial version? Do you think we could re-use some of that code? Do we need to add another base class for 3D peak maps?

Yes the code is in the initial version. We could adopt the VLine Plot to color lines with intensity values. However, the figure itself needs to be initialized differently as a 3D figure.

@axelwalter axelwalter merged commit 6fe4712 into OpenMS:main Jul 31, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants