Skip to content

Conversation

@djkapner
Copy link
Contributor

@djkapner djkapner commented Oct 9, 2020

This is the PR branch for the 2.4.0 release candidate of AllenSDK. The release date is tentatively ~2020-10-29.

  • Need to reach a consensus on what to do about argschema update before merging this into master

Doug Ollerenshaw and others added 3 commits June 8, 2020 21:23
raster_plot now correctly plots each unit once.  A unit
test checks that the number of plotted lines equals the
number of units passed to the raster_plot function.
@codecov-io
Copy link

codecov-io commented Oct 9, 2020

Codecov Report

Merging #1744 (0fd2426) into master (71281bf) will increase coverage by 0.21%.
The diff coverage is 56.10%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #1744      +/-   ##
==========================================
+ Coverage   36.23%   36.44%   +0.21%     
==========================================
  Files         346      350       +4     
  Lines       33824    33881      +57     
==========================================
+ Hits        12255    12348      +93     
+ Misses      21569    21533      -36     
Impacted Files Coverage Δ
allensdk/brain_observatory/behavior/validation.py 0.00% <0.00%> (ø)
allensdk/internal/api/mtrain_api.py 0.00% <0.00%> (ø)
allensdk/brain_observatory/session_api_utils.py 62.01% <12.72%> (-36.64%) ⬇️
...apis/data_transforms/behavior_ophys_data_xforms.py 32.92% <21.56%> (ø)
...ry/behavior/session_apis/data_io/ophys_lims_api.py 45.12% <45.12%> (ø)
...behavior/session_apis/data_io/behavior_lims_api.py 47.00% <47.00%> (ø)
...or/session_apis/data_io/behavior_ophys_json_api.py 51.47% <51.47%> (ø)
...or/session_apis/data_io/behavior_ophys_lims_api.py 60.00% <60.00%> (ø)
.../project_apis/data_io/behavior_project_lims_api.py 59.57% <62.50%> (ø)
.../behavior/session_apis/abcs/behavior_ophys_base.py 71.73% <71.42%> (ø)
... and 38 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 71281bf...0fd2426. Read the comment docs.

@njmei njmei changed the title begins new release candidate branch rc/2.4.0 Oct 9, 2020
@wbwakeman wbwakeman added the noprogram not directly related to an Insitute program label Oct 15, 2020
djkapner and others added 18 commits October 15, 2020 10:10
This commit removes fields that already exist as part of the pyNWB
"Subject" class from the custom behavior ophys subject extension.
Previously dff and fluorescence traces were being stored as
ROIs x timepoints Numpy arrays. This commit implements pyNWB
best practice by storing the data with the time dimension
first.
This is a combination of the following 3 commits:

1) Simplify custom behavior ophys pyNWB extension

   Previously, the OphysBehaviorMetadata extension included a number
   of fields that already exist in base pyNWB classes. This commit
   moves "emission_lambda", "excitation_lambda", "indicator",
   "location", "session_start_time", and "imaging_rate" fields so
   that they are stored in pyNWB classes like ImagingPlane and
   OpticalChannel. This greatly simplifies the OphysBehaviorMetadata
   extension definition.

2) Add more detailed "doc" description to pyNWB extension fields

3) Address pyNWB NWBFile.modules deprecation warning

   Trying to access pyNWB processing modules via the "NWBFile.modules"
   attribute is now deprecated in favor of "NWBFile.processing".
   This commit implements the favored method for accessing
   pyNWB processing modules from an NWBFile object.
Previously, ROIs were added to the behavior ophys NWB file
incorrectly. ROIs all ended up starting at the 0,0 corner of the
whole imaging space.

This was caused by feeding the "LIMS formatted ROI" (List[List[bool]])
through the allensdk.brain_observatory.roi_masks.create_roi_mask
function twice instead of only once. This happened once in
allensdk.internal.api.ophys_lims_api.OphysLimsApi
get_cell_specimen_table() method and yet again in the
allensdk.brain_observatory.nwb.__init__ add_cell_specimen_table()
function.

Now, only the OphysLimsApi get_cell_specimen_table() is the only
place where LIMS formatted ROIs are deserialized to the
allensdk.brain_observatory.roi_masks.RoiMask class.

A CellSpecimenTable schema discrepancy was also fixed.
'x' and 'y' fields should be Int not Float based on what the
behavior_ophys_write_nwb_strategy from LIMS is passing in.
This commit adds experiment descriptions to be havior ophys NWB files.
These descriptions can be determined based on the behavior ophys
"session type" string. Example:

"OPHYS_1_images_A" is the first stage of the visual behavior experiments
"OPHYS_6_images_A" is the last stage of the visual behavior experiments
GH #1739 - Fix ecephys raster_plot plotting units twice
…hys-nwb-issues

1733 address behavior ophys nwb issues
…l-join-ids

Pull data for summary of experiments/sessions even if there are nulls in join ids
- Adds the ability to pass kwargs to the RmaEngine
including the timeout kwarg
- Adds pytests to verify the kwargs are passed into
the RmaEngine
- Updates the default timeout for the HttpEngine from
10 to 50 minutes
njmei and others added 28 commits November 23, 2020 00:47
Previously, the behavior_ophys_nwb_api.py file implemented
an `equals` function for comparing two Session objects as well as
a `compare_fields` function for checking that attributes of
Session objects are also equal.

Both these functions have been moved to:
allensdk.brain_observatory.session_api_utils

Additional, the `equals` function has been renamed to `sessions_are_equal`
and the `compare_fields` function has been renamed to
`compare_session_fields`.
Renames the `data_fetchers` directory to `data_io`. Data API
implementations for sessions and/or project in cases like NWB
not only 'fetch' data but also 'save' so 'fetcher' would be
too narrow a description.

This applies to:
- allensdk.brain_observatory.behavior.project_apis.data_fetchers
- allensdk.brain_observatory.behavior.session_apis.data_fetchers
This commit will rename the BehaviorDataSession class located in
allensdk.brain_observatory.behavior.behavior_data_session
to just BehaviorSession (and
allensdk.brain_observatory.behavior.behavior_session).

This is to bring the naming in line with the BehaviorOphysSession class
and also to draw a distinction between a 'session' class that
end-users will interact with and a 'data io API' class that loads
session data from data sources like LIMS/NWB.
This commit moved behavior and behavior + ophys session
LIMS API related classes from `allensdk.internal.api` to:

`allensdk.brain_observatory.behavior.session_apis.data_io`

The files moved include:
- behavior_data_lims_api: Newer version API for behavior only data from LIMS
- behavior_lims_api: Older version API for behavior only data from LIMS
- behavior_ophys_api: API for behavior + ophys data from LIMS
- ophy_lims_api: API for ophys data from LIMS
It turns out that "BehaviorDataLimsApi" located in
allensdk.brain_observatory.behavior.session_apis.data_io.behavior_data_lims_api
(previously allensdk.internal.api.behavior_data_lims_api)
and the "BehaviorLimsApi" located in
allensdk.brain_observatory.bheavior.session_apis.data_io.behavior_lims_api
(previously allensdk.internal.api.behavior_lims_api)
are historically separate implementations of what *should* be the same class.

This commit does the following:
- Add the ability to construct a BehaviorDataLimsApi from a foraging_id.
  This was basically the only unique functionality provided by the
  BehaviorLimsApi class.
- Remove the BehaviorLimsApi class completely.
- Spruce up how database connections are created
- Rename the BehaviorDataLimsApi to BehaviorLimsApi
- Rename behavior_data_lims_api.py to behavior_lims_api.py
This commits fixes imports of the BehaviorOphysLimsApi
which previously lived at:
allensdk.internal.api.behavior_ophys_api.py

Its new location is:
allensdk.brain_observatory.behavior.session_apis.data_io.behavior_ophys_lims_api.py
This commit fixes imports for the OphysLimsApi that previously lived at:
allensdk.internal.api.ophys_lims_api.py

It now lives at:
allensdk.brain_observatory.behavior.session_apis.data_io.ophys_lims_api.py
The commit moves the BehaviorOphysJsonApi defined in
allensdk.brain_observatory.behavior.write_nwb.__main__ to
allensdk.brain_observatory.behavior.session_apis.data_io.behavior_ophys_json_api.

This commit also fixes a bug with multiple inheritence and method
resolution order. ABC classes should always be inherited last so that
the first parent of a class will be able to fill the subclass with
defined methods (and not abstract methods).
This commit factors out 'data only' (LIMS/BehaviorOphysJson/NWB) API
methods from methods that are required by the OphysSession class.

This commit creates a new set of classes (BehaviorDataXforms, BehaviorOphysDataXforms)
that transforms data obtained from 'data only' methods into sources for
the OphysSession class.
This commits fixes an inheritance order issue that was causing tests to
fail. Specifically:

- The BehaviorLimsApi inherits from BehaviorDataXforms so
  BehaviorOphysDataXforms should always be inherited before
  BehaviorLimsApi

- The OphysLimsApi methods should be used before BehaviorLimsApi methods

- The BehaviorOphysLimsApi class should not be calling super() in its
  __init__. Rather it should set and provide self.ophys_experiment_id,
  self.behavior_session_id, self.lims_db, and self.mtrain_db
This commit aims to improve the clarity of documentation
for the db_connection_creator helper function. This includes
changing the 'default_credentials' param to 'fallback_credentials'.
This commit standardizes the formatting of LIMS queries
for the various behavior APIs.

This commit also adds better documentation API methods
(including return type and better descriptions of
data retrieved by queries).
This is a combination of the following two commits:

1) Add more documentation

   This commit adds more documentation to brain_observatory.behavior
   data api methods.

2) Fix a couple of LIMS query typos

   A set of typos where a semicolon was introduced within a single quote
   block resulted in a non-functional query. This commit fixes these
   typos.
…ys-inheritence

1769 modify behavior ophys inheritence
Fix typing.Protocol import in Python 3.8
@djkapner djkapner merged commit 9f64b72 into master Dec 21, 2020
@djkapner djkapner deleted the rc/2.4.0 branch January 4, 2021 15:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

noprogram not directly related to an Insitute program

Projects

None yet

Development

Successfully merging this pull request may close these issues.

9 participants