Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[#12048] Patch Usage Statistics Migration #12889

Merged
merged 9 commits into from
Mar 18, 2024

Conversation

ziqing26
Copy link
Contributor

Part of #12048

Outline of Solution

  • To migrate newly generated usage statistics after the last run of the script, use a START_TIME benchmark to migrate those entities generated after this benchmark time.
  • Requires user to run a query SELECT MAX(start_time) FROM usage_statistics; and then copy paste the string to the START_TIME_STRING. (Is it possible to do the query in the script?)

@ziqing26 ziqing26 added the s.Ongoing The PR is being worked on by the author(s) label Mar 14, 2024
@FergusMok
Copy link
Contributor

One problem with the patching right now is that we still have to scan through all the datastore entities, and only filter in memory. This is because filtering in datastore requires we create an index.

Probably not needed for now since the number of entities is relatively small, and scanning datastore is relatively fast, but it may be needed next time for course entities.

CriteriaBuilder cb = HibernateUtil.getCriteriaBuilder();
CriteriaQuery<Instant> cq = cb.createQuery(Instant.class);
Root<UsageStatistics> root = cq.from(UsageStatistics.class);
cq.select(cb.greatest(root.<Instant>get("createdAt")));
Copy link
Contributor

@NicolasCwy NicolasCwy Mar 18, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this should be changed to startTime since createdAt is what time the entity is created in the SQL DB. UsageStatistics are created one per hour, so this startTime should be unique as well

@NicolasCwy NicolasCwy added s.ToMerge The PR is approved by all reviewers including final reviewer; ready for merging and removed s.Ongoing The PR is being worked on by the author(s) labels Mar 18, 2024
Copy link
Contributor

@NicolasCwy NicolasCwy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@FergusMok FergusMok merged commit 2eedf1d into TEAMMATES:master Mar 18, 2024
11 checks passed
@cedricongjh cedricongjh added this to the V9.0.0-beta.1 milestone Mar 24, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
s.ToMerge The PR is approved by all reviewers including final reviewer; ready for merging
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants