Error with co-expression modules

[cpalmateer]

Hi,

After running GRNboost, exporting the matrix, and bringing it back in with:
adjacencies.to_csv(ADJACENCIES_FNAME, index=False, sep='\t')
adjacencies = pd.read_csv(ADJACENCIES_FNAME, sep='\t')

I tried to run:
modules = list(modules_from_adjacencies(adjacencies, ex_matrix))

I was met with this error:
TypeError Traceback (most recent call last)
in
----> 1 modules = list(modules_from_adjacencies(adjacencies, ex_matrix))

~/.local/lib/python3.6/site-packages/pyscenic/utils.py in modules_from_adjacencies(adjacencies, ex_mtx, thresholds, top_n_targets, top_n_regulators, min_genes, absolute_thresholds, rho_dichotomize, keep_only_activating, rho_threshold, rho_mask_dropouts)
262 LOGGER.info("Calculating Pearson correlations.")
263 adjacencies = add_correlation(adjacencies, ex_mtx,
--> 264 rho_threshold=rho_threshold, mask_dropouts=rho_mask_dropouts)
265 activating_modules = adjacencies[adjacencies[COLUMN_NAME_REGULATION] > 0.0]
266 if keep_only_activating:

~/.local/lib/python3.6/site-packages/pyscenic/utils.py in add_correlation(adjacencies, ex_mtx, rho_threshold, mask_dropouts)
127 if mask_dropouts:
128 ex_mtx = ex_mtx.sort_index(axis=1)
--> 129 col_idx_pairs = _create_idx_pairs(adjacencies, ex_mtx)
130 rhos = masked_rho4pairs(ex_mtx.values, col_idx_pairs, 0.0)
131 else:

~/.local/lib/python3.6/site-packages/pyscenic/utils.py in _create_idx_pairs(adjacencies, exp_mtx)
69 # Create sorted list of genes that take part in a TF-target link.
70 genes = set(adjacencies.TF).union(set(adjacencies.target))
---> 71 sorted_genes = sorted(genes)
72
73 # Find column idx in the expression matrix of each gene that takes part in a link. Having the column index of genes

TypeError: '<' not supported between instances of 'str' and 'float'

Any assistance would be great!

Thank you,

Colleen

[cflerin]

Hi Colleen,

Are you working with Drosophila data by any chance? The nan gene was causing a similar str-float problem for us in the past. This was fixed in release 0.9.7 however. If not, it could be an issue with the feather databases you're using. Can you share the version of pyscenic you're using and the details for the support files (databases, motifs, etc)?

Chris

[cpalmateer]

Hi Chris,

I'm running pyscenic==0.9.9 with dm6-5kb-upstream-full-tx-11species.mc8nr.feather.
So it looks like I'm using version 8 of the feather database. Would using version 9 fix the problem?

Thank you,

Colleen

[cpalmateer]

After filtering my expression matrix more stringently, I was able to work around this problem. However, I can't seem to get past the modules. Now it throws the error:

/home/lab1/.local/lib/python3.6/site-packages/pyscenic/utils.py:138: RuntimeWarning: invalid value encountered in greater
regulations = (rhos > rho_threshold).astype(int) - (rhos < -rho_threshold).astype(int)
/home/lab1/.local/lib/python3.6/site-packages/pyscenic/utils.py:138: RuntimeWarning: invalid value encountered in less
regulations = (rhos > rho_threshold).astype(int) - (rhos < -rho_threshold).astype(int)

[bramvds]

Hi Colleen,

These are just warnings. Reason is masking of dropouts when calculating correlations between genes: for some gene pairs not many samples remain.

Kindest regards,
Bram