Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Programming exercises: Cache dependencies in the Java blackbox template #8351

Merged
merged 28 commits into from
Apr 22, 2024

Conversation

BaumiCoder
Copy link
Member

@BaumiCoder BaumiCoder commented Apr 8, 2024

Checklist

General

Server

Client

No Changes at the client

Changes affecting Programming Exercises

  • High priority: I tested all changes and their related features with all corresponding user types on a test server configured with the integrated lifecycle setup (LocalVC and LocalCI). Only applicable to the GitLab/Jenkins template. The Aeolus/LocalCI one does not support setting Docker run flags specific to the exercise.
  • I tested all changes and their related features with all corresponding user types on a test server configured with Gitlab and Jenkins.

Motivation and Context

The actual caching approach with Docker volumes has one mature disadvange.
It needs to have a testing run for the solution before a testing run for a student submission
on each host, which is running the containers.
That is a problem if you need more hosts to run the container because of scaling issues.

The new approach uses a dedicated cache server (Maven proxy) that does not have this limitation.

Description

  1. Switch the blackbox Dejagnu template to cache Maven dependencies with Sonatype Nexus.
  2. Document how to use the Sonatype Nexus Caching for Maven and Gradle.
  3. Document how to restrict / disable the network access for student submission corresponding to the used caching approach.

Steps for Testing

Prerequisites:

  • 1 Instructor
  • 1 Student
  • A course where the student is enrolled / can enrol
  1. Log in to Artemis with the Instructor account
    1. Navigate to a course
    2. Create a new Programming Exercise there
      1. Choose language Java
      2. Choose type DejaGnu
    3. Check if the tests for the solution repository are 100 %
  2. Log in to Artemis with the Student account
    1. Go the new Programming Exercise
    2. Make a submission that should pass a test (or all) -- You could use the contents from the solution repository.

The updated documentation can be found at https://artemis-platform--8351.org.readthedocs.build/en/8351/admin/setup/programming-exercises.html#caching-maven-dependencies-with-sonatype-nexus

Testserver States

Note

These badges show the state of the test servers.
Green = Currently available, Red = Currently locked







Review Progress

Performance Review

  • I (as a reviewer) confirm that the server changes (in particular related to database calls) are implemented with a very good performance

Code Review

  • Code Review 1
  • Code Review 2

Manual Tests

  • Test 1
  • Test 2

Test Coverage

Only trivial change in java code.

BaumiCoder and others added 23 commits April 5, 2024 15:05
.mvn is falsely detected as a file to copy.
The solution with the docker volumns is not necessary anymore
with the maven proxy approach. Uncomment with some explanation
if somebody prefer this caching approach.
Link to documentation, where more details will be available.
Last option needs further system configuration,
so not active by default. Mention docs for further information.
Change to method name "setup()" for fitting with template using this
artemis/src/main/resources/templates/jenkins/java/blackbox/regularRuns/pipeline.groovy
Have to be called "code-block" instead of "code"
+ correct ident
+ restore caption build.gradle
Sphinx does print listitems with subitems complitly bold,
so switch to italic to emphasize.
There was a empty line missing above the section header,
because of the line above, which was added in the Review process.
@BaumiCoder BaumiCoder requested a review from a team as a code owner April 8, 2024 18:04
Copy link

coderabbitai bot commented Apr 8, 2024

Walkthrough

The update focuses on enhancing Maven dependency handling and security in programming exercises setup. It includes instructions for caching Maven dependencies via Sonatype Nexus, security considerations for Jenkins setups, and updates to directory filtering and Maven configuration for improved performance and security measures.

Changes

Files Change Summary
docs/admin/setup/programming-exercises.rst,
src/main/java/de/tum/in/www1/artemis/service/FileService.java
Added detailed instructions for caching Maven dependencies with Sonatype Nexus, security considerations for Jenkins, and updated directory filtering. Included ".mvn" in IGNORED_DIRECTORY_SUFFIXES list.
src/main/resources/templates/java/test/.../.mvn/local-settings.xml,
src/main/resources/templates/java/test/.../.mvn/maven.config
Introduced Maven settings files for configuring Artemis Cache mirror and specifying local Maven settings.
src/main/resources/templates/jenkins/.../pipeline.groovy Updated setup function with comments and logic for Docker and Maven flags based on repository type.

Possibly related issues

  • Exam: Divide the Testing Process into Steps #6621: The changes in this PR, particularly the update to include ".mvn" in ignored directory suffixes and the introduction of Maven settings for better dependency management, could indirectly support the objectives of ensuring proper test case handling and compilation steps by potentially streamlining the setup process and reducing manual intervention. This could make the environment more predictable and manageable, indirectly aiding in the issue's objectives of improving test case management during exams.

Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

Share
Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>.
    • Generate unit testing code for this file.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai generate unit testing code for this file.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai generate interesting stats about this repository and render them as a table.
    • @coderabbitai show all the console.log statements in this repository.
    • @coderabbitai read src/utils.ts and generate unit testing code.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (invoked as PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger a review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai help to get help.

Additionally, you can add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.

CodeRabbit Configration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

@github-actions github-actions bot added server Pull requests that update Java code. (Added Automatically!) documentation template labels Apr 8, 2024
@BaumiCoder
Copy link
Member Author

Corresponding to the Exam Mode parts of the Pull Request template: Does
"affecting Programming Exercises" also apply for this Pull Request (i.e. only a template change)?

b-fein
b-fein previously approved these changes Apr 9, 2024
Copy link
Contributor

@b-fein b-fein left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code/documentation review: Looks good apart from two minor things. 👍

docs/admin/setup/programming-exercises.rst Outdated Show resolved Hide resolved
docs/admin/setup/programming-exercises.rst Outdated Show resolved Hide resolved
@b-fein b-fein changed the title Programming exercises: Show new caching approach in Artemis Dejagnu templates and documente approach Programming exercises: Document new Maven dependency caching approach and apply it to the Java blackbox template Apr 9, 2024
Co-authored-by: Benedikt Fein <fein@fim.uni-passau.de>
coderabbitai[bot]
coderabbitai bot previously approved these changes Apr 10, 2024
Co-authored-by: Benedikt Fein <fein@fim.uni-passau.de>
Copy link

There hasn't been any activity on this pull request recently. Therefore, this pull request has been automatically marked as stale and will be closed if no further activity occurs within seven days. Thank you for your contributions.

Copy link
Contributor

@pzdr7 pzdr7 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

demonstrated during testing session; LGTM

Copy link
Contributor

@coolchock coolchock left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tested during testing session, works as described

Copy link
Contributor

@egekurt123 egekurt123 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Manually tested during testing session on legacy ts2, works as expected

Copy link
Contributor

@undernagruzez undernagruzez left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tested during testing session, works as described

@b-fein b-fein added this to the 7.0.1 milestone Apr 22, 2024
@krusche krusche modified the milestones: 7.0.1, 7.0.2 Apr 22, 2024
@krusche krusche changed the title Programming exercises: Document new Maven dependency caching approach and apply it to the Java blackbox template Programming exercises: Cache dependencies in the Java blackbox template Apr 22, 2024
@krusche krusche merged commit 75a3df9 into develop Apr 22, 2024
38 of 43 checks passed
@krusche krusche deleted the feature/Dejagnu-templates-caching branch April 22, 2024 19:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
Archived in project
Development

Successfully merging this pull request may close these issues.

None yet

7 participants