Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

update photosynthesis package to use xarray and analyze entire induction curve protocols #790

Merged
merged 177 commits into from
Jul 22, 2021

Conversation

dschneiderch
Copy link
Collaborator

@dschneiderch dschneiderch commented Jun 18, 2021

This pull request implements the new data model for psII measurements and associated analyze_* functions.
We are using a new class to store multiple data arrays. Each data array has 4 dimensions (x, y, frame_label, measurement). Each data array must have a predefined name: darkadapted, lightadapted, rgb, chlorophyll, anthocyanin

[x] PSII_data class to store multiple xarray data arrays
[x] read_cropreporter to import data from cropreporter instrument
[] read_walz to import data from walz imaging pam
[x] analyze_yii to analyze photosynthetic efficiency for any number of measurements
[x] analyze_npq to analyze nonphotochemical quenching for any number of measurements
[x] tests
[x] documentation

Type of update
Is this a:

  • New feature or feature enhancement
  • Work in progress

Associated issues
closes #636

Additional context
Add any other context about the problem here.

update the way INF file is located. make it more agnostic to binary image type
switch strategies, use INF file as input and find all image data that corresponds
comment out my code
start adding the code for creating xarray objects
update transpose to be appropriate for x-array now instead of numpy
now making x-arrays that have parameter labels embedded
overhaul the way data is formatted after being read in
replace with debug method
extract fmax for plotting
update docs and other code to make more consistent with pcv.readimage
other misc updates to comments
overhaul function to accept x-array data from read_cropreporter
first draft at writing new NPQ analysis function
update outputs names and observation data getting stored
update observations
correct source function in observations
mixed up .DAT and .INF
@dschneiderch
Copy link
Collaborator Author

dschneiderch commented Jul 16, 2021

here's an issue from 2015(!) Don't squeeze DataArray before plotting

@nfahlgren
Copy link
Member

Using FacetGrid directly seems to work for both one and multiple measurements:

gd = FacetGrid(data=fvfm, col="measurement", col_wrap=1)
gd.map_dataarray(xr.plot.imshow, "x", "y")

We could put this directly in the YII and NPQ functions for now and not use the _debug module.

@dschneiderch
Copy link
Collaborator Author

@nfahlgren next level wizardry!
another idea I had overnight is to check whether dims x and y are present and if so call pcolormesh(). i actually think they do something like this under the hood but then squeeze the dims too early. I think this would cover most use cases.

@dschneiderch
Copy link
Collaborator Author

also, for hte col_wrap arg I suggest col_wrap=int(np.ceil(fvfm.measurement.size/3)) where the number is the max # of columns

@dschneiderch
Copy link
Collaborator Author

this will all fail if the col arg is removed from this test because then _show_dataarray() returns a histogram as a tuple which does not have self.fig.savefig()

should we just always limit to pcolormesh()? what does a line plot return from xarray?

def test_plantcv_print_image_dataarray():
    da = psii_cropreporter('darkadapted').squeeze('measurement', drop=True)
    try:
        #this fails without col or row as an arg because xarray will create a matplotlib histogram that comes back as a tuple and then print_image will try to use self.fig.savefig which doesn't exist for a tuple
        pcv.print_image(img=da, col='frame_label', filename='/dev/null') 
    except RuntimeError:
        assert False
    # Assert that the image was plotted without error
    assert True

Dominik Schneider and others added 2 commits July 19, 2021 12:38
Timepoint frame labels other than Fm/Fmp, F0, Fp, Fdark/Flight were None
@dschneiderch
Copy link
Collaborator Author

@nfahlgren I think I may have broken the debug= functionality again. sometimes I see the plots even when I have debug=None or debug='print'. clearly I don't understand something.

however, in terms of plotting data array data, I think it will be best to limit to facetgrid functionality using pcolormesh(). it greatly simplifies print_image and plot_image from what I can tell.
the plot output from xarray is different based on the number of dimensions:
1D: da.plot() as a list len 1 containing matplotlib.lines.Line2D bc xarray makes a line plot
3+D: da.plot() comes back as a tuple bc xarray makes a histogram unless col and/or row is supplied for facetting
2D: da.plot() will make an image. facetted images if extra dims are specified as kwargs col or row. in practice I found it's easy to lose track of this and provide too many or not enough dims so I explicitly call pcolormesh() with some sanity checks/error messages in a new private module _show_dataarray . I also check for x and y because I think thats the main use case

@nfahlgren
Copy link
Member

I agree, limiting to pcolormesh makes sense for our usecase

Copy link
Member

@nfahlgren nfahlgren left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Alright, I think everything is wrapped up! Thanks for the huge effort on this @dschneiderch and @HaleySchuhl!

@nfahlgren nfahlgren merged commit 2c4dc01 into 4.x Jul 22, 2021
PlantCV4 automation moved this from In progress to Done Jul 22, 2021
@nfahlgren nfahlgren deleted the march21-update-crop_reporter-multiplemeasurements branch July 22, 2021 05:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
new feature New feature ideas and solutions ready to review
Projects
PlantCV4
  
Done
Development

Successfully merging this pull request may close these issues.

Photosynthesis analysis quality filtering
3 participants