Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Don't depend on DataCite for MDC stats #618

Closed
7 tasks done
sfisher opened this issue Jan 8, 2020 · 0 comments
Closed
7 tasks done

Don't depend on DataCite for MDC stats #618

sfisher opened this issue Jan 8, 2020 · 0 comments
Assignees
Labels
MDC Make Data Count

Comments

@sfisher
Copy link
Contributor

sfisher commented Jan 8, 2020

Problem description, and links to relevant helpdesk tickets:

DataCite doesn't store our reports and we can't query them, so we want to go ahead and pretend they don't exist and just put our stats into our totals in the database.

Describe the solution you'd like:

  • Process stats every weekend
    • We may need to process part of 1 or 2 months, depending on whether the previous week straddles the month's break.
    • Don't submit to DataCite since it doesn't work.
    • Save the updated output report JSON files to a known place along with all the other past json report files we've made previously.
  • Repopulate the database totals from our reports
    • Since we don't really know the time from that previous totals in the database represent and impossible to match up with reports that keep getting changed, then just remove all totals every week and re-process all of them into the totals.
    • The rake task for this is called counter:cop_manual, takes the rails environment and json file directory. It will then read through these files and populate stats into the database.

I want to test this out before we do it on production, so I think I need to get the whole python environment set up on stage. Stage is load balanced so it has logs on 2 servers like production (unlike dev). So stage is the best place to test the script.

**Additional context: **

@sfisher sfisher self-assigned this Jan 8, 2020
@sfisher sfisher added the MDC Make Data Count label Jan 8, 2020
@sfisher sfisher closed this as completed Aug 17, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
MDC Make Data Count
Projects
None yet
Development

No branches or pull requests

1 participant