Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

k_selection_plot and consensus - TypeError: H should have the same dtype as X. #8

Closed
d-laub opened this issue Jul 7, 2020 · 10 comments

Comments

@d-laub
Copy link

d-laub commented Jul 7, 2020

Hello,

I'm encountering the following error while running k_selection_plot.

I'm working with RNA-Seq data from multiple ~homogeneous tissue samples, rather than scRNA-Seq.

Here are my commands leading up to calling k_selection_plot:

python $CNMF prepare --output-dir $OUTPUT_DIR \
--name $OUTPUT_PREFIX \
-c $COUNTS \
--tpm $TPM \
-k $K_LIST \
--n-iter $N_ITER \
--total-workers $TOTAL_WORKERS \
--seed $SEED \
--densify

# This ran as a SLURM array job
python $CNMF factorize --output-dir $OUTPUT_DIR \
--name $OUTPUT_PREFIX \
--worker-index $WORKER_ID

python $CNMF combine --output-dir $OUTPUT_DIR \
--name $OUTPUT_PREFIX

And here is the traceback for the error after the call:

python $CNMF k_selection_plot --output-dir $OUTPUT_DIR \
--name $OUTPUT_PREFIX
Traceback (most recent call last):
  File "/cellar/users/dlaub/.programs/cnmf/cnmf.py", line 883, in <module>
    cnmf_obj.k_selection_plot()
  File "/cellar/users/dlaub/.programs/cnmf/cnmf.py", line 696, in k_selection_plot
    stats.append(self.consensus(k, skip_density_and_return_after_stats=True).stats)
  File "/cellar/users/dlaub/.programs/cnmf/cnmf.py", line 556, in consensus
    nmf_kwargs=refit_nmf_kwargs)
  File "/cellar/users/dlaub/.programs/cnmf/cnmf.py", line 410, in _nmf
    (usages, spectra, niter) = non_negative_factorization(X, **nmf_kwargs)
  File "/cellar/users/dlaub/anaconda3/envs/py3/lib/python3.7/site-packages/sklearn/utils/validation.py", line 73, in inner_f
    return f(**kwargs)
  File "/cellar/users/dlaub/anaconda3/envs/py3/lib/python3.7/site-packages/sklearn/decomposition/_nmf.py", line 1044, in non_negative_factorization
    "{}.".format(H.dtype))
TypeError: H should have the same dtype as X. Got H.dtype = float64.

Could I get some help with this? Thanks!

@d-laub
Copy link
Author

d-laub commented Jul 7, 2020

I think the --densify flag might be causing the issue, since when I run try code that loads the counts and tpm matrices into scanpy objects, the dtype of X is float32 (the default for AnnData()).

Further, when I reimport the data from the .h5ad object they are written to in save_norm_counts() their datatype changes to NumPy's '<f8'.

I edited the consensus function within cnmf.py to cast the dtype of norm_counts.X to 'float64' and k_selection_plot ran without errors.

@d-laub d-laub closed this as completed Jul 7, 2020
@d-laub d-laub changed the title TypeError: H should have the same dtype as X. k_selection_plot - TypeError: H should have the same dtype as X. Jul 7, 2020
@d-laub d-laub changed the title k_selection_plot - TypeError: H should have the same dtype as X. k_selection_plot and consensus - TypeError: H should have the same dtype as X. Jul 9, 2020
@d-laub d-laub changed the title k_selection_plot and consensus - TypeError: H should have the same dtype as X. k_selection_plot and consensus - TypeError: H should have the same dtype as X. Jul 9, 2020
@d-laub
Copy link
Author

d-laub commented Jul 9, 2020

I had another similar issue calling consensus (weirdly only for some runs and not others). On line 580 in the original cnmf.py I checked if tpm.X.dtype is 'float64' and changed it to 'float64' otherwise and it works now.

@Pentayouth
Copy link

In case someone doesn't know
add
norm_counts.X = norm_counts.X.astype(np.float64)
at cnmf.py line 503
will solve this

@jasminelmah
Copy link

I did run into this issue for step 4 of the README tutorial, and adding the line suggested by Pentayouth worked. I've now run into what seems to be a related issue, for step 5 of the README. I am doing a test run with the PBMC example data from Seurat, which I first converted to an h5ad file before going through the README.

Command: python3 ./cnmf.py consensus --output-dir ./example_data --name example_cNMF --components 10 --local-density-threshold 0.01 --show-clustering

Traceback:

(base) vpn1722517526:example_PBMC jasmineliannemah$ python3 ./cnmf.py consensus --output-dir ./example_data --name example_cNMF --components 10 --local-density-threshold 0.01 --show-clustering
/Library/Python/3.7/site-packages/sklearn/utils/deprecation.py:143: FutureWarning: The sklearn.decomposition.nmf module is deprecated in version 0.22 and will be removed in version 0.24. The corresponding classes / functions should instead be imported from sklearn.decomposition. Anything that cannot be imported from sklearn.decomposition is now part of the private API.
warnings.warn(message, FutureWarning)
Traceback (most recent call last):
File "./cnmf.py", line 882, in
cnmf_obj.consensus(k, args.local_density_threshold, args.local_neighborhood_size, args.show_clustering)
File "./cnmf.py", line 558, in consensus
nmf_kwargs=refit_nmf_kwargs)
File "./cnmf.py", line 410, in _nmf
(usages, spectra, niter) = non_negative_factorization(X, **nmf_kwargs)
File "/Library/Python/3.7/site-packages/sklearn/utils/validation.py", line 72, in inner_f
return f(**kwargs)
File "/Library/Python/3.7/site-packages/sklearn/decomposition/_nmf.py", line 1044, in non_negative_factorization
"{}.".format(H.dtype))
TypeError: H should have the same dtype as X. Got H.dtype = float64.

It has been a while since I've worked with Python so I'm struggling to understand how to edit the code. Thanks for any help!

@avishai987
Copy link

avishai987 commented Nov 29, 2022

In case someone doesn't know add norm_counts.X = norm_counts.X.astype(np.float64) at cnmf.py line 503 will solve this

Hi, could it be that it's not the right line number? now it seems to raise name error on norm_counts.

P.S: I added it at the beginning of factorize method, and now it works.

@qinting
Copy link

qinting commented Dec 15, 2022

Hi, I encountered the same issue using the most recent version of cnmf. I also tried all suggestions listed above, however, the problem still existed. Could anyone please let me know how to solve the problem? Where exactly I should add norm_counts.X = norm_counts.X.astype(np.float64) in cnmf.py?

@ShaoboKang
Copy link

Hi,I want to know where exactly are you add norm_counts.X = norm_counts.X.astype(np.float64) in cnmf.py?

@avishai987
Copy link

Hi,I want to know where exactly are you add norm_counts.X = norm_counts.X.astype(np.float64) in cnmf.py?

Hi, I encountered the same issue using the most recent version of cnmf. I also tried all suggestions listed above, however, the problem still existed. Could anyone please let me know how to solve the problem? Where exactly I should add norm_counts.X = norm_counts.X.astype(np.float64) in cnmf.py?

Find cnmf.py. At the beginning of the factorize method ( def factorize(self,worker_i=0, total_workers=1,):), one of the parameters is "norm_counts ". Insert "norm_counts.X = norm_counts.X.astype(np.float64)" as the first line of the method, so it will fix norm_counts.

@ShaoboKang
Copy link

ShaoboKang commented Apr 10, 2023 via email

@honghh2018
Copy link

Hi everyone,
I get the same error in cnmf 1.3.4
i follow above solution, but it still get error
i add the norm_counts.X = norm_counts.X.astype(np.float64) in cnmf.py below picture

1702461193627

and how to fix this error?
Best
hanhuihong

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

7 participants