Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

plantuml-server 1.2021.12 -> 1.2022.2 #163431

Merged
merged 2 commits into from
Mar 14, 2022

Conversation

nazarewk
Copy link
Member

@nazarewk nazarewk commented Mar 9, 2022

Description of changes

Suplements #163416 in updating plantuml suite to consistent versions.

There were following issues resolved on the way:

  • plantuml-server required 11+ version of JDK, updated the module to keep consistent versions,
  • maven was not configured for reproducible builds and embedded timestamps of files in *.war output package
  • plantuml-server started requiring Jetty 11+ (due to javax being renamed to jakarta for legal reasons)
Things done
  • Built on platform(s)
    • x86_64-linux
    • aarch64-linux
    • x86_64-darwin
    • aarch64-darwin
  • For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
  • Tested, as applicable:
  • Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
  • Tested basic functionality of all binary files (usually in ./result/bin/):
    1. I've reconstructed the services.plantuml-server command from this branch results:
      nix build '.#plantuml-server' '.#jdk' '.#jetty'
      "${PWD}/result-1/bin/java" -jar "${PWD}/result-2/start.jar" --module=deploy,http,jsp jetty.home="${PWD}/result-2" jetty.base="${PWD}/result" jetty.http.host=127.0.0.1 jetty.http.port=7890
      
    2. Confirmed it's working by going to http://127.0.0.1:7890/plantuml in the browser
  • 22.05 Release Notes (or backporting 21.11 Release notes)
    • (Package updates) Added a release notes entry if the change is major or breaking
    • (Module updates) Added a release notes entry if the change is significant
    • (Module addition) Added a release notes entry if adding a new NixOS module
    • (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes
  • Fits CONTRIBUTING.md.

@nazarewk

This comment was marked as outdated.

@truh truh self-requested a review March 9, 2022 12:31
@truh
Copy link
Member

truh commented Mar 9, 2022

@nazarewk I encountered this problem as well when I tried to update this package. My workaround was to use a different jdk jdk17_headless #159063 based on https://stackoverflow.com/questions/63154787/maven-is-not-using-java-11-fatal-error-compiling-invalid-target-release-11. It's possible that such a change would require updating the jre used in the plantuml-server module to a newer java version as well.

I didn't go through with it since the output hash of deps kept changing.

@drupol
Copy link
Contributor

drupol commented Mar 9, 2022

How about changing in the nativeBuildInputs: jdk8_headless in jdk17_headless ? Can you give it a try ?

Also, is there any kind of documentation that says what is the required version of the JDK for this?

@nazarewk
Copy link
Member Author

Also, is there any kind of documentation that says what is the required version of the JDK for this?

Looks like it requires JDK 11+ and Maven 3.0.2+ https://github.com/plantuml/plantuml-server/blob/0330b7c4a20292082465c021ec415cc013167998/README.md?plain=1#L27-L30

@nazarewk
Copy link
Member Author

@truh @drupol looks like it works with jdk17_headless, but just like truh said, the output is not stable

@drupol
Copy link
Contributor

drupol commented Mar 10, 2022

Why not using jdk11?

@nazarewk
Copy link
Member Author

Why not using jdk11?

This is the same for jdk11_headless

@drupol
Copy link
Contributor

drupol commented Mar 10, 2022

Why not using jdk11?

This is the same for jdk11_headless

Then maybe we should open an issue upstream ?

@nazarewk
Copy link
Member Author

Why not using jdk11?

This is the same for jdk11_headless

Then maybe we should open an issue upstream ?

I mean the output is unstable for both jdk11_headless and jdk17_headless

@nazarewk nazarewk marked this pull request as draft March 10, 2022 10:19
@nazarewk
Copy link
Member Author

nazarewk commented Mar 10, 2022

Looks like timestamps in output *.war are differing:

nazarewk@nazarewk-krul /tmp/plantuml-server % ls -la            
total 38560
drwxr-xr-x  4 nazarewk users      120 Mar 10 11:39 .
drwxrwxrwt 27 root     root       780 Mar 10 11:38 ..
drwxr-xr-x  5 nazarewk users      200 Mar 10 11:40 plantuml-1
-r--r--r--  1 nazarewk users 19740424 Mar 10 11:34 plantuml-1.war
drwxr-xr-x  5 nazarewk users      200 Mar 10 11:40 plantuml-2
-r--r--r--  1 nazarewk users 19740424 Mar 10 11:36 plantuml-2.war
nazarewk@nazarewk-krul /tmp/plantuml-server % ls -la plantuml-1
total 20
drwxr-xr-x 5 nazarewk users  200 Mar 10 11:40 .
drwxr-xr-x 4 nazarewk users  120 Mar 10 11:39 ..
-rw-r--r-- 1 nazarewk users 1884 Mar 10 10:05 error.jsp
-rw-r--r-- 1 nazarewk users 1150 Mar 10 10:05 favicon.ico
-rw-r--r-- 1 nazarewk users  123 Mar 10 10:05 footer.jspf
-rw-r--r-- 1 nazarewk users 3870 Mar 10 10:05 index.jsp
drwxr-xr-x 3 nazarewk users   80 Mar 10 11:40 META-INF
-rw-r--r-- 1 nazarewk users  947 Mar 10 10:05 plantuml.css
drwxr-xr-x 2 nazarewk users  120 Mar 10 10:05 resource
drwxr-xr-x 4 nazarewk users  100 Mar 10 10:05 WEB-INF
nazarewk@nazarewk-krul /tmp/plantuml-server % ls -la plantuml-2
total 20
drwxr-xr-x 5 nazarewk users  200 Mar 10 11:40 .
drwxr-xr-x 4 nazarewk users  120 Mar 10 11:39 ..
-rw-r--r-- 1 nazarewk users 1884 Mar 10 10:36 error.jsp
-rw-r--r-- 1 nazarewk users 1150 Mar 10 10:36 favicon.ico
-rw-r--r-- 1 nazarewk users  123 Mar 10 10:36 footer.jspf
-rw-r--r-- 1 nazarewk users 3870 Mar 10 10:36 index.jsp
drwxr-xr-x 3 nazarewk users   80 Mar 10 11:40 META-INF
-rw-r--r-- 1 nazarewk users  947 Mar 10 10:36 plantuml.css
drwxr-xr-x 2 nazarewk users  120 Mar 10 10:36 resource
drwxr-xr-x 4 nazarewk users  100 Mar 10 10:36 WEB-INF
nazarewk@nazarewk-krul /tmp/plantuml-server % diff -r plantuml-* 
diff: extra operand 'plantuml-2'
diff: Try 'diff --help' for more information.
2 nazarewk@nazarewk-krul /tmp/plantuml-server % diff -r plantuml-{1,2}

@nazarewk
Copy link
Member Author

Looks like there is reproducible-build-maven-plugin, but i have no clue whether it's possible or how to apply it to this use case https://zlika.github.io/reproducible-build-maven-plugin/

@nazarewk nazarewk marked this pull request as ready for review March 10, 2022 10:54
@nazarewk
Copy link
Member Author

FYI: -Dproject.build.outputTimestamp=0 to mvn package was enough to trigger reproducible builds as per https://maven.apache.org/guides/mini/guide-reproducible-builds.html#how-to-test-my-maven-build-reproducibility

@nazarewk nazarewk removed the request for review from truh March 10, 2022 14:21
@Artturin
Copy link
Member

Has this been tested if so I will merge

@nazarewk
Copy link
Member Author

nazarewk commented Mar 10, 2022

looks like this actually doesn't work yet due to switching from javax to jakarta and nixpkgs Jetty not keeping up with it (nixpkgs has version 9 while 11+ is required):

Copy link
Contributor

@emmanuelrosa emmanuelrosa left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are three stable versions of Jetty: 9.4, 10, and 11.
Since there are no other packages in Nixpkgs which use Jetty, I'm OK with upgrading the Jetty package to version 11.

If other packages arise in the future needing version 9.4 or 10, I'll deal with it then.

Copy link
Contributor

@drupol drupol left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@truh
Copy link
Member

truh commented Mar 10, 2022

I did a little bit of testing. When I started the service from this branch SVG generation didn't work. I was able to fix the problem by replacing jdk17_headless with jdk in nixos/modules/services/web-apps/plantuml-server.nix.

The error is NoClassDefFoundError: Could not initialize class net.sourceforge.plantuml.ugraphic.UFont

My test setup:

  • Create configuration.nix
{ pkgs, ... }:

{
  services.plantuml-server = {
    enable = true;
    listenHost = "0.0.0.0";
    listenPort = 28080;
  };
  users.users.root.password = "hunter2";
  users.mutableUsers = false;
  services.qemuGuest.enable = true;
  environment.systemPackages = with pkgs; [ lynx ];
}

  • Run:
nixos-rebuild build-vm --fast -I nixos-config=./configuration.nix -I nixpkgs=.
result/bin/run-nixos-vm
  • Inside the VM, after login (root:hunter2) run:
lynx localhost:28080/plantuml/

@nazarewk
Copy link
Member Author

The error is NoClassDefFoundError: Could not initialize class net.sourceforge.plantuml.ugraphic.UFont

Yeah, i finished yesterday with the same error, thanks for tracking it down.

updated because plantuml-server requires Jetty 11+
- changed JDK version to 17 (11+ required)
- added maven build reproducibility
@nazarewk
Copy link
Member Author

I have tested it with jdk by reconstructing the commandline from services.plantuml-server, I'm on flake based system and those instructions above to spawn a VM gave me very weird results (it had GDM and hostname the same as my host system).

@lukegb lukegb merged commit 247a180 into NixOS:master Mar 14, 2022
@nazarewk nazarewk deleted the update-plantuml-server branch March 14, 2022 13:39
@vcunat
Copy link
Member

vcunat commented Mar 15, 2022

This broke the build on Hydra, I think: https://hydra.nixos.org/eval/1748939?filter=plantuml#tabs-now-fail

@nazarewk
Copy link
Member Author

This broke the build on Hydra, I think: hydra.nixos.org/eval/1748939?filter=plantuml#tabs-now-fail

Any idea what is wrong? I have trouble pinpointing the issue (responsible logs or anything)

@vcunat
Copy link
Member

vcunat commented Mar 15, 2022

Locally I'm getting

hash mismatch in fixed-output derivation '/nix/store/7wvrgia71mrr658rpxak3dw6aszqid0b-plantuml-server-1.2022.2-deps':
  wanted: sha256:1042nkxic732ppjma2lc894c8812h4js8dijm73k05v3gq2q45q2
  got:    sha256:1vc2j6xk1csagbsx15qdgxsglhmm75rsj8npjf19qj0i8k8jd46y
cannot build derivation '/nix/store/mllvsyizjl5frnwpsic5gp1msivxz0n5-plantuml-server-1.2022.2.drv': 1 dependencies couldn't be built

@nazarewk
Copy link
Member Author

what system are you on?

@vcunat
Copy link
Member

vcunat commented Mar 15, 2022

x86_64 NixOS. This build was nix build -f. plantuml-server on commit c0ede26 specifically.

@nazarewk
Copy link
Member Author

nazarewk commented Mar 15, 2022

It works just fine for me on the same system using nix build '.#plantuml-server' and latest master (fad085c08f97ff526cda9bcfb83c6c00291c66fd)

looks like i did not in fact have latest master

@truh
Copy link
Member

truh commented Mar 15, 2022

@nazarewk did it rebuild the deps for you or were the deps still in your nix store?

@nazarewk
Copy link
Member Author

I've sha256-3pAm0UQRSJyCk9ciqXM5tUL6dH8Nl9D1ekqzMLuRgu0= on 2 conscutive rebuild with nix-collect-garbage -d in between, which is not what was in the PR.

Might be that dependencies are not pinned in plantuml-server and get different results in different times?

@truh
Copy link
Member

truh commented Mar 15, 2022

I think it's likely that the maven artifacts in the repository are mutable.

@truh
Copy link
Member

truh commented Mar 15, 2022

Maybe it would be better to just download the war file from the Github release and call it a day.

nazarewk added a commit to nazarewk/nixpkgs that referenced this pull request Mar 15, 2022
after merging NixOS#163431 `plantuml-server-*-deps` derivation turned out unstable, this commit switches to fetching `plantuml.war` directly from releases
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

7 participants