Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Programming exercises: Speedup archival #7165

Merged
merged 10 commits into from
Sep 11, 2023
Merged

Conversation

krusche
Copy link
Member

@krusche krusche commented Sep 9, 2023

Archiving programming exercises as part of a course or exam was done sequentially which can take quite some time, especially in courses with between 10.000 and 80.000 repositories (such as on the TUM production system).

This PR speeds up the process by running the clone and zip operation in 10 parallel threads. In a local test, the archival time of one course was around 5-7x faster with this approach.

Checklist

General

Server

  • Important: I implemented the changes with a very good performance and prevented too many (unnecessary) database calls.
  • I followed the coding and design guidelines.
  • I documented the Java code using JavaDoc style.

Steps for Testing

Prerequisites:

  • Archive a course (and an exam) with passed end date on the course management detail page
  • Download the archive and verify all student repositories are included

Code Review

  • Code Review 1
  • Code Review 2

Manual Tests

  • Test 1
  • Test 2

@krusche krusche requested a review from a team as a code owner September 9, 2023 17:27
@krusche krusche added this to the 6.4.3 milestone Sep 9, 2023
@github-actions github-actions bot added tests server Pull requests that update Java code. (Added Automatically!) labels Sep 9, 2023
Strohgelaender
Strohgelaender previously approved these changes Sep 9, 2023
Copy link
Contributor

@Strohgelaender Strohgelaender left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

code lgtm

@krusche krusche merged commit 1ddc676 into develop Sep 11, 2023
21 of 25 checks passed
@krusche krusche deleted the feature/speedup-archive branch September 11, 2023 08:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ready for review server Pull requests that update Java code. (Added Automatically!) tests
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants