-
Notifications
You must be signed in to change notification settings - Fork 9
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Spike: is it possible to reconstitute the current set of evaluations by ingesting the current real world #3234
Comments
We ran this for all 7 PCI groups:
resulted in ~327 evaluations being correctly recorded. |
To look into a local database:
|
To count unique evaluations for a group on a copy of the production database:
|
The Arcadia group has 2642 evaluations specified on their group card in prod. Local, fresh ingestion yielded 2083. We believe that this is correlated with switching off the old Arcadia Hypothesis group, see 6a6137a. In another experiment we tried ingesting from both Hypothesis groups with a cutoff date between the two. We saw 1475 evaluations ingested from the current group (after 2023-04-15), and 556 ingested from the old group. We discovered we did a backfill around 2023-08-01 that accidentally recorded all the content from the new group, creating duplicate evaluations:
|
The preLights group has 1175 evaluations specified on their group card in prod. Local, fresh ingestion yielded 1542. |
The prereview group has 415 evaluations specified on their group card in prod. Local, fresh ingestion yielded 405. |
It is known NCRC does not have an ingestion set up currently, as a dormant group; we would need to bring that back. |
The eLife group has 26996 evaluations specified on their group card in prod. Local, fresh ingestion yielded 24750 (but with 1666 lefts). |
To compare, the current Hypothesis group we ingest from for eLife:
which is about ~500 less than the group card in prod. The number of unique evaluation locators recorded in prod is:
which suggests we erased/removed almost ~2000 of them over time? |
More on eLife suggests the Hypothes.is group is the source of truth and will cover our use case:
|
The Rapid Reviews Infectious Diseases group has 995 evaluations specified on their group card in prod. Local, fresh ingestion yielded only 114 even after unhardcoding Edit: various problems in the ingestion:
After hardcoding |
No description provided.
The text was updated successfully, but these errors were encountered: