Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

tsne won't work with sklearn.prepare #233

Closed
isinyaaa opened this issue Nov 29, 2022 · 1 comment
Closed

tsne won't work with sklearn.prepare #233

isinyaaa opened this issue Nov 29, 2022 · 1 comment
Assignees

Comments

@isinyaaa
Copy link

isinyaaa commented Nov 29, 2022

When running the same steps with the same data on colab I get pretty good results with tsne, but locally (probably because of the Python version) I'm not able to run pyLDAvis.sklearn.prepare as I get ValueError: perplexity must be less than n_samples.

I know that colab is running 3.7 and locally I got 3.10 . I also know that both use pyLDAvis version 3.3.1 so its probably broken because of a Scikit update.

I was able to get it to work by manually setting a perplexity value in the TSNE object initialization under pyLDAvis/_prepare.py but it sure isn´t optimal.

Error log
Traceback (most recent call last):
  File "/home/isinyaaa/projects/foss-gpgpu-stack/analyze.py", line 418, in <module>
    main(args)
  File "/home/isinyaaa/projects/foss-gpgpu-stack/analyze.py", line 347, in main
    args.workers).run(vectorizer, processed_data)
  File "/home/isinyaaa/projects/foss-gpgpu-stack/analyze.py", line 128, in run
    self.save_result_as_html(model, data, vectorizer)
  File "/home/isinyaaa/projects/foss-gpgpu-stack/analyze.py", line 148, in save_result_as_html
    super().save_result_as_html(prepare, model, data, vectorizer, mds='tsne')
  File "/home/isinyaaa/projects/foss-gpgpu-stack/analyze.py", line 111, in save_result_as_html
    LDAvis_prepared = prepare(*args, **kwargs)
  File "/home/isinyaaa/.local/lib/python3.10/site-packages/pyLDAvis/sklearn.py", line 95, in prepare
    return pyLDAvis.prepare(**opts)
  File "/home/isinyaaa/.local/lib/python3.10/site-packages/pyLDAvis/_prepare.py", line 443, in prepare
    topic_coordinates = _topic_coordinates(mds, topic_term_dists, topic_proportion, start_index)
  File "/home/isinyaaa/.local/lib/python3.10/site-packages/pyLDAvis/_prepare.py", line 192, in _topic_coordinates
    mds_res = mds(topic_term_dists)
  File "/home/isinyaaa/.local/lib/python3.10/site-packages/pyLDAvis/_prepare.py", line 167, in js_TSNE
    return model.fit_transform(dist_matrix)
  File "/home/isinyaaa/.local/lib/python3.10/site-packages/sklearn/manifold/_t_sne.py", line 1122, in fit_transform
    self._check_params_vs_input(X)
  File "/home/isinyaaa/.local/lib/python3.10/site-packages/sklearn/manifold/_t_sne.py", line 793, in _check_params_vs_input
    raise ValueError("perplexity must be less than n_samples")
ValueError: perplexity must be less than n_samples
@msusol
Copy link
Collaborator

msusol commented Feb 11, 2023

see #239
see #235

@msusol msusol closed this as completed Feb 11, 2023
@msusol msusol self-assigned this Feb 11, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants