Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

How to empty the dictionary quickly #125

Closed
FrankDataAnalystPython opened this issue Jul 1, 2022 · 3 comments
Closed

How to empty the dictionary quickly #125

FrankDataAnalystPython opened this issue Jul 1, 2022 · 3 comments

Comments

@FrankDataAnalystPython
Copy link

Dear Sir:
I am just wondering is there a way to empty the .words dict quickly in an inexpensive way after loading dictionary.

I am facing a case where I need to constantly update the dictionary of the symspell, but it is very slow for each initialization

Regards

@FrankDataAnalystPython
Copy link
Author

Dear Sir:
I am also wondering, is there a way to just focus on a subset of the symspell.words for calculating the distance? Many Thanks

Regards

@mammothb
Copy link
Owner

mammothb commented Jul 4, 2022

I am just wondering is there a way to empty the .words dict quickly in an inexpensive way after loading dictionary.

I am facing a case where I need to constantly update the dictionary of the symspell, but it is very slow for each initialization

The words and other related data are found within these member variables and are implemented as dictionary. So I think you can probably just use the Python built-in dictionary clear() function.

I am also wondering, is there a way to just focus on a subset of the symspell.words for calculating the distance?

You'll have to write your custom functions/methods for this. You can perhaps try to load a subset of the dictionary words before running lookup methods.

To make switching of different dictionary "subset" quicker, I think you can save the words and related member variables as pickles and then just load them instead of having to parse them again. You can refer to the _load_pickle_stream() method to see which member variables should be saved and loaded

@FrankDataAnalystPython
Copy link
Author

Many Thanks!!!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants