Check if word exists #24
Conversation
…ction-ml#23) * remove dependency on google storage * Modified return of LocalFasttextPlayer, added time and code fields * Modifications for better separation of settings and code * Better division of server vs local games * Moved players into the main func * delete empty server_settings.py from server folder * Further separation of GAME_SCOPES * Separation of flask_app players and Hat Game players
There was a problem hiding this comment.
Good work, @naidenovaleksei! 👍
I left some comments, let's discuss them.
the_hat_game/game.py
Outdated
| self.used_words = corpus_to_words(corpus_path) | ||
| elif vocab_path is not None: | ||
| with open(vocab_path, encoding="utf-8") as f: | ||
| words = f.readlines() |
There was a problem hiding this comment.
Let's use another variable instead of words here, because we use this name for another param in __init__.
There was a problem hiding this comment.
fixed in 26e6f0d
the_hat_game/game.py
Outdated
| existing_words = [w for w in words if len(wordnet.synsets(w)) > 0] | ||
| def remove_non_existing_words(self, words): | ||
| if self.used_words is not None: | ||
| existing_words = [w for w in words if w in self.used_words] |
There was a problem hiding this comment.
There could be a situation when initial words you pass to __init__ aren't all there in self.used_words. I.e. we guess the word which is not in self.used_words and will be removed. Let's add words to self.used_words if the later is not empty (in __init__ method).
There was a problem hiding this comment.
Maybe this will fix this strange behaviour, but I would check if that's not wordnet.synsets filtering out those words, and can we fix it with using nltk.corpus.words.
There was a problem hiding this comment.
Let's add words to self.used_words if the later is not empty (in init method).
Oh, sure. Now it seems like a bug. Thank you!
There was a problem hiding this comment.
Let's add words to self.used_words if the later is not empty (in init method).
added in 26e6f0d
There was a problem hiding this comment.
Maybe this will fix this strange behaviour
yes, adding words for guessing to whitelist fixed this strange behaviour
I would check if that's not wordnet.synsets filtering out those words, and can we fix it with using nltk.corpus.words
wordnet.synsets really filter rare English words, but nltk.corpus.words do the same (see #1 (comment))
the_hat_game/game.py
Outdated
| @@ -13,6 +13,7 @@ | |||
| import the_hat_game.nltk_setup # noqa: F401 | |||
| from the_hat_game.loggers import c_handler, logger | |||
| from the_hat_game.players import RemotePlayer | |||
| from data.utils import corpus_to_words | |||
There was a problem hiding this comment.
data/ migrated to server/ folder. The idea was to split the logic into two parts (the-hat-game part - only game logic, don't know anything about server implementation; server - everything about working with cloud, uploading texts, etc).
So maybe we should move some data functions to the-hat-game/data.py. Those which can be used in the-hat-game and aren't related to server directly. Namely, sent_2_words, corpus_to_words, and maybe save_words (in this case we should remove default vocab_path arg value, because it's related to server implementation).
There was a problem hiding this comment.
fixed in 0481ab4
sent_2_words, corpus_to_words were added, save_words not (it is not used anywhere)
Try to solve #1
Details:
wordnet.synsetshas been saved to check if word exist due to its using in other code (see link)nltk.corpus.wordsisn't used to avoid another one dependency in nltk (nltk.download("words")will be required here)vocab_pathfor reading all used English words from vocabularycorpus_pathfor building vocabulary of all used words