Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make OJS2 to OJS3 metrics upgrade faster on large installations #4131

Closed
ajnyga opened this Issue Oct 10, 2018 · 3 comments

Comments

Projects
None yet
3 participants
@ajnyga
Copy link
Contributor

ajnyga commented Oct 10, 2018

Discussion here: https://forum.pkp.sfu.ca/t/ojs-upgrade-from-3-0-2-to-3-1-0-1-stuck-on-slow-query/36706

The single sql query used to update the assoc_id's of assoc_type 531 is extremely slow in larger metric tables. For example a 900 megabyte metrics table takes around 34 hours to run the given query.

The query should be broken into smaller parts using an upgrade function. Pull request soon.

@ajnyga

This comment has been minimized.

Copy link
Contributor Author

ajnyga commented Oct 10, 2018

ping @bozana and @asmecher
I have used this to upgrade a 2 gigabyte metrics table with around 160 000 531's and 7000 supplementary file hits. This took a little over an hour. I am running a new test with the same database right now.

@ajnyga

This comment has been minimized.

Copy link
Contributor Author

ajnyga commented Oct 11, 2018

The second test run with the 2 gigabyte metrics table now took 1.3 hours. The whole upgrade around 9 hours (not sure what the exact time was since it was finished during the night). The whole database was around 3.2 gigabytes.

For reference a 2 gigabyte db with 1 gigabyte metrics table took 34 hours with the old code.

@bozana bozana added this to the OJS/OMP 3.2 milestone Oct 11, 2018

@bozana bozana added the Enhancement label Oct 11, 2018

ajnyga added a commit to ajnyga/ojs that referenced this issue Oct 11, 2018

bozana added a commit to pkp/ojs that referenced this issue Oct 11, 2018

Merge pull request #2131 from ajnyga/fix-4131
pkp/pkp-lib#4131 use function to update 531 assoc_id's in metrics table
@bozana

This comment has been minimized.

Copy link
Collaborator

bozana commented Oct 11, 2018

merged, thus closing....

@bozana bozana closed this Oct 11, 2018

@asmecher asmecher modified the milestones: OJS/OMP 3.2, OJS/OMP 3.1.2 Dec 21, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.