Mice with no training history

[alexpiet]

Time-boxed investigation for a maximum of 2 days to see if the fix to #1666 addresses this, or to suggest additional work that may be needed to address it.

Describe the bug
The behavior session table is a data frame that includes the full training history for each mouse. Some mice; however, only have OPHYS sessions, and do not have any TRAINING stages.

To Reproduce
Two mice that I found randomly (donor_id = 744911447, and 756577240). You can load the behavior_session_table, and then filter for that donor_id, and see that the only session types are
OPHYS, and no TRAINING stages.

behavior_sessions = cache. get_behavior_session_table()
behavior_sessions.query('donor_id == 756577240')

Expected behavior
Mice cannot go to ophys without training, so I expect to see training data for each mouse.

Environment (please complete the following information):

• OS & version: CentOS 7
• Python version 3
• AllenSDK version v1.6.0

Additional context

1. I haven't tracked down other mice that show this issue.
2. I wonder if the problem is a donor_id vs specimen_id issue?

@wbwakeman @dougollerenshaw @matchings

[dougollerenshaw]

To aid in troubleshooting, I generated a list of all training sessions for ophys production mice here: '/allen/programs/braintv/workgroups/nc-ophys/visual_behavior/production_behavior_sessions.2020.04.08.csv'

This should make it possible to check that the SDK is returning the full training history for a given mouse.

[alexpiet]

Of 75 Scientific mice, 15 have no training data.

[alexpiet]

@dougollerenshaw How did you create the csv file above? There are mice in the SDK cache, that I believe are production mice that arent in your file. For example the session in this screenshot:

[dougollerenshaw]

Hmm, I'm not quite sure what went wrong. I'd used the visual_behavior get_qc_cache function, which has since been replaced with get_visual_behavior_cache. The session you referenced above is there. I resaved a CSV here:
'/allen/programs/braintv/workgroups/nc-ophys/visual_behavior/production_behavior_sessions.2020.05.12.csv'

I verified that all mice that have at least on session in the visual_behavior_cache also have at least one session in this table.

[alexpiet]

Thanks @dougollerenshaw

I can now confirm that the behavior_sessions table does not have sessions listed in Doug's csv file, for at least some mice.

[alexpiet]

Doing a systematic check, I find 744 sessions that are missing from the behavior_session_table, from 26 mice. All of the mice have donor_ids starting with 7, so they are all older mice.

Comparison script is at /allen/programs/braintv/workgroups/nc-ophys/alex.piet/QC/check_behavior_sessions.py

Dataframe containing all missing sessions at:
/allen/programs/braintv/workgroups/nc-ophys/alex.piet/QC/missing_sessions.py

If you try to load these sessions from the cache:
cache.get_behavior_session_data(str(missing.behavior_session_id[0])) you get a "pandas.errors.EmptyDataError: No columns to parse from file"

[alexpiet]

Here is a full traceback on the error when I load the sessions:

[dougollerenshaw]

@alexpiet I don't really understand the underlying error, but it looks like some problem with the caching mechanism.

As a workaround, you can load directly from LIMS. I just confirmed that this works for the session in your traceback:

from allensdk.brain_observatory.behavior.behavior_data_session import BehaviorDataSession
behavior_session_id = 700070730
session = BehaviorDataSession.from_lims(behavior_session_id)

I can get the lick dataframe for this session:
session.licks

But I'm getting errors on some other attributes. For example:

session.metadata

---------------------------------------------------------------------------
OneResultExpectedError                    Traceback (most recent call last)
<ipython-input-36-dcbc02333a10> in <module>
----> 1 session.metadata

~/Code/AllenSDK/allensdk/brain_observatory/behavior/behavior_data_session.py in metadata(self)
    235         """
    236         if self._metadata is None:
--> 237             self._metadata = self.api.get_metadata()
    238         return self._metadata
    239 

~/Code/AllenSDK/allensdk/internal/api/behavior_data_lims_api.py in get_metadata(self)
    489             bs_uuid = uuid.UUID(self.get_behavior_session_uuid())
    490         metadata = {
--> 491             "rig_name": self.get_rig_name(),
    492             "sex": self.get_sex(),
    493             "age": self.get_age(),

~/Code/AllenSDK/allensdk/api/cache.py in wrapper(*args, **kwargs)
     73         if result is not sentinel:
     74             return result
---> 75         result = f(*args, **kwargs)
     76         cache[key] = result
     77         return result

~/Code/AllenSDK/allensdk/internal/api/behavior_data_lims_api.py in get_rig_name(self)
    379             WHERE bs.id = {self.behavior_session_id};
    380         """
--> 381         return self.lims_db.fetchone(query, strict=True)
    382 
    383     @memoize

~/Code/AllenSDK/allensdk/internal/api/__init__.py in fetchone(self, query, strict)
     44 
     45     def fetchone(self, query, strict=True):
---> 46         response = one(list(self.select(query).to_dict().values()))
     47         if strict is True and (len(response) != 1 or response[0] is None):
     48             raise OneResultExpectedError

~/Code/AllenSDK/allensdk/__init__.py in one(x)
     60         return x
     61     if xlen != 1:
---> 62         raise OneResultExpectedError('Expected length one result, received: {} results from query'.format(x))
     63     if isinstance(x, set):
     64         return list(x)[0]

OneResultExpectedError: Expected length one result, received: [] results from query

[kschelonka]

Thanks Doug. These data are not in the expected format for the stimuli (missing 'images' key) and don't seem to have running data, among other issues (like you've mentioned above). That's probably what's bubbling up to the cache and causing issues.

[dougollerenshaw]

Thanks @kschelonka.

I'm assuming this is the running issue you're referring to: #1559

And these are the image issues:
#1558
#1557