error reproducing notebook

[ltosti]

Hi there,

I tried to reproduce the pancreas Jupyter notebook (planning to eventually add some more data to it). However, after loading the 4 datasets in the holder, when I run adata = holder[0].concatenate(holder[1:], join='outer') I get the following error:

<ipython-input-27-8bddebd9bb8e> in <module>
----> 1 adata = holder[0].concatenate(holder[1:], join='outer')
      2 #adata.X = adata.X.tocsr()
      3 #adata = adata[:,['ERCC' not in item.upper() for item in adata.var_names]]
      4 #adata.raw = sc.pp.log1p(adata, copy=True)
      5 #sc.pp.normalize_per_cell(adata, counts_per_cell_after=1e4)

/anaconda3/envs/leiden/lib/python3.6/site-packages/anndata/base.py in concatenate(self, join, batch_key, batch_categories, index_unique, *adatas)
   1807                 # constructed like that
   1808                 X[obs_i:obs_i+ad.n_obs,
-> 1809                   var_names.isin(vars_intersect)] = ad[:, vars_intersect].X
   1810             else:
   1811                 Xs.append(ad[:, vars_intersect].X)

/anaconda3/envs/leiden/lib/python3.6/site-packages/anndata/base.py in __getitem__(self, index)
   1299     def __getitem__(self, index):
   1300         """Returns a sliced view of the object."""
-> 1301         return self._getitem_view(index)
   1302 
   1303     def _getitem_view(self, index):

/anaconda3/envs/leiden/lib/python3.6/site-packages/anndata/base.py in _getitem_view(self, index)
   1302 
   1303     def _getitem_view(self, index):
-> 1304         oidx, vidx = self._normalize_indices(index)
   1305         return AnnData(self, oidx=oidx, vidx=vidx, asview=True)
   1306 

/anaconda3/envs/leiden/lib/python3.6/site-packages/anndata/base.py in _normalize_indices(self, index)
   1278                 return index
   1279         obs, var = super(AnnData, self)._unpack_index(index)
-> 1280         obs = _normalize_index(obs, self.obs_names)
   1281         var = _normalize_index(var, self.var_names)
   1282         return obs, var

/anaconda3/envs/leiden/lib/python3.6/site-packages/anndata/base.py in _normalize_index(index, names)
    238     if not isinstance(names, RangeIndex):
    239         assert names.dtype != float and names.dtype != int, \
--> 240             'Don’t call _normalize_index with non-categorical/string names'
    241 
    242     # the following is insanely slow for sequences, we replaced it using pandas below

AssertionError: Don’t call _normalize_index with non-categorical/string names

These are the current versions I'm using:
scanpy==1.3.6 anndata==0.6.13 numpy==1.15.3 scipy==1.1.0 pandas==0.23.4 scikit-learn==0.20.0 statsmodels==0.9.0 python-igraph==0.7.1 louvain==0.6.1.

Do you know what is causing this?

Thank you!

[ktpolanski]

Hm, that's quite a bit newer than the setup I've used, and it's not an actual BBKNN error. You're probably better off asking the scanpy/anndata guys, sorry.

[ktpolanski]

Actually, seeing how this isn't at all crucial to the BBKNN part of the example, you can skip down to In[16]: or so by downloading ftp://ngs.sanger.ac.uk/production/teichmann/BBKNN/objects-pancreas.zip and going adata = sc.read('objects-pancreas/pancreas.h5ad'). That should do the trick and allow you to do BBKNN things.

[ktpolanski]

Heh, I updated my setup on one of my OpenStack instances and bumped into the same thing doing something else. Google pointed me towards this: scverse/anndata#77

And it actually works, as silly as it is. As such, I've stuck some .copy() on the affected AnnDatas and it started to work again

[ltosti]

Thank you @ktpolanski ; so when you append new data to the holder, you use .copy()?

[ktpolanski]

While my context was different, in this case here you could try adding holder[-1] = holder[-1].copy() at the end of each code block that processes the R bits. That should probably do it.

[ltosti]

Thank you, but unfortunately this doesn't seem to work in this case. I'm waiting for @falexwolf and @flying-sheep to help me with the error!