-
Notifications
You must be signed in to change notification settings - Fork 15
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Create "Core" Table Once and Then Never Again #15
Comments
As part of this, we should also have a REGEX on the core table to get rid of punctuation and contractions. |
Also, do we even need a top_friends_table core table? Could we just query directly every time? Querying directly might make filtering easier. Filters we probably want: is_from_me, group_message, person, date. |
As part of this ticket, I think I'll change it so that there is no concept of a Emojis, Word, or Friends table. Instead there are just core tables and we query off of those core tables every time. |
We only need to create the
SPLIT_TEXT_TABLE
once and then never again. This should be a "Core" table that is created on initialize. This will greatly improve performance because then the WordCount and EmojiCount (and I imagine a word by day chart) tables can just read from this core table, which is created only once.The core table could also already exclude stopWords, unicode characters, etc, so we don't have to run those WHERE clauses multiple times.
The text was updated successfully, but these errors were encountered: