Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Normalize Events and Histories #213

Closed
mfenner opened this issue Jan 2, 2015 · 0 comments
Closed

Normalize Events and Histories #213

mfenner opened this issue Jan 2, 2015 · 0 comments
Assignees
Labels

Comments

@mfenner
Copy link
Member

@mfenner mfenner commented Jan 2, 2015

This will make it easier to properly display events associated with an article, but also to slice and dice our data as needed for reporting. Normalizing events and histories also makes the push API easier to use, as there is a clear data model behind it. Another important consequence is that we no longer need CouchDB and can store all data in MySQL.

  • store events as individual database rows
  • using the works table, and the events table (see above) to describe the relationship (e.g. Cites, Reviews)
  • normalize events using CSL (Citation Style Language) for easier display in aggregate form and easier import/export
  • store raw json response in separate responses table with 1-1 relationship to works table (for faster queries of the works table)
  • store counts over time in two tables: months and days. These tables are used both for sources that have events (e.g. twitter), and sources that don't (e.g. counter).
  • event counts are aggregated in the retrieval_statuses table
  • categories (e.g. html, pdf, comments, likes) are stored in separate fields in the retrieval_statuses, months and days tables to allow sorting and filtering by category
@mfenner mfenner added the roadmap label Jan 2, 2015
@mfenner mfenner self-assigned this Jan 2, 2015
mfenner pushed a commit that referenced this issue Jan 16, 2015
mfenner pushed a commit that referenced this issue Jan 16, 2015
mfenner pushed a commit that referenced this issue Jan 16, 2015
mfenner pushed a commit that referenced this issue Jan 16, 2015
mfenner pushed a commit that referenced this issue Jan 16, 2015
mfenner pushed a commit that referenced this issue Jan 16, 2015
mfenner pushed a commit that referenced this issue Jan 16, 2015
mfenner pushed a commit that referenced this issue Jan 16, 2015
mfenner pushed a commit that referenced this issue Jan 16, 2015
mfenner pushed a commit that referenced this issue Jan 16, 2015
mfenner pushed a commit that referenced this issue Jan 16, 2015
mfenner pushed a commit that referenced this issue Feb 5, 2015
mfenner pushed a commit that referenced this issue Mar 23, 2015
mfenner pushed a commit that referenced this issue Mar 23, 2015
mfenner pushed a commit that referenced this issue Mar 23, 2015
mfenner pushed a commit that referenced this issue Mar 23, 2015
mfenner pushed a commit that referenced this issue Mar 23, 2015
mfenner pushed a commit that referenced this issue Mar 23, 2015
mfenner pushed a commit that referenced this issue Mar 23, 2015
mfenner pushed a commit that referenced this issue Mar 24, 2015
mfenner pushed a commit that referenced this issue Apr 19, 2015
Martin Fenner
mfenner pushed a commit that referenced this issue Apr 19, 2015
mfenner pushed a commit that referenced this issue Apr 19, 2015
mfenner pushed a commit that referenced this issue Apr 20, 2015
mfenner pushed a commit that referenced this issue Apr 27, 2015
mfenner pushed a commit that referenced this issue Apr 27, 2015
mfenner pushed a commit that referenced this issue Apr 30, 2015
mfenner pushed a commit that referenced this issue May 2, 2015
Martin Fenner
mfenner pushed a commit that referenced this issue May 3, 2015
mfenner pushed a commit that referenced this issue May 4, 2015
mfenner pushed a commit that referenced this issue May 4, 2015
mfenner pushed a commit that referenced this issue May 4, 2015
mfenner pushed a commit that referenced this issue May 4, 2015
Martin Fenner
mfenner pushed a commit that referenced this issue May 4, 2015
mfenner pushed a commit that referenced this issue May 4, 2015
Martin Fenner
mfenner pushed a commit that referenced this issue May 4, 2015
mfenner pushed a commit that referenced this issue May 5, 2015
mfenner pushed a commit that referenced this issue May 5, 2015
Martin Fenner
mfenner pushed a commit that referenced this issue May 5, 2015
Martin Fenner
mfenner pushed a commit that referenced this issue May 14, 2015
mfenner pushed a commit that referenced this issue May 14, 2015
mfenner pushed a commit that referenced this issue May 15, 2015
mfenner pushed a commit that referenced this issue May 15, 2015
mfenner pushed a commit that referenced this issue May 16, 2015
mfenner pushed a commit that referenced this issue May 16, 2015
mfenner pushed a commit that referenced this issue May 16, 2015
@mfenner mfenner closed this Jun 14, 2015
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
1 participant
You can’t perform that action at this time.