New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[CM H2] 3713 java binding maven to gradle migration #3714
[CM H2] 3713 java binding maven to gradle migration #3714
Conversation
Something went wrong in the rebase. Probably cherry-picking you own commits is the easiest way to resolve the problem here. |
1cfb352
to
2c0469d
Compare
19e2659
to
92a62b5
Compare
src/bindings/jna/libelektra/src/main/java/org/libelektra/plugin/NativePlugin.java
Outdated
Show resolved
Hide resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I found some minor things in an initial review.
5436c90
to
22ad591
Compare
The following issue may be closed when this PR is merged: #3269 |
https://build.libelektra.org/blue/organizations/jenkins/libelektra/detail/PR-3714/43/pipeline failed with non-zero exit value 134 on `fedora-32-full´. Seems like a resource problem (e.g. insufficient RAM) to me. Any inputs? |
jenkins build libelektra please |
I cannot read from the output that it is about too little RAM*, SIGSEGV can be anything. As it happens in ksRewind, maybe we did something wrong in the JNA binding? Btw. just because the tests run locally does not mean that there isn't a memory bug. Did you try to run with valgrind? *Actually the machines have plenty of RAM (even i7 has 8GB). |
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Great job! So good to finally see this arriving.
@@ -157,25 +157,38 @@ you up to date with the multi-language support provided by Elektra. | |||
|
|||
### JNA | |||
|
|||
Since internal iterator support for `KeySet` is due to being dropped, the following methods have been removed: | |||
- Since internal iterator support for `KeySet` is due to being dropped, the following methods have been removed: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You can also make the improvements of Java Bindings+publishing to maven a highlight.
### <<Binding2>> | ||
- Renamed `KeyException` specializations: `KeyInvalidNameException`, `KeyTypeConversionException`, `KeyTypeMismatchException` | ||
|
||
- Ongoing work on bringing the JNA binding up to scratch and improving developer experience. Both for JNA binding API consumers, as well as future JNA binding contrubutors. _(Michael Tucek)_ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Put this to a new section Outlook below.
src/bindings/jna/libelektra/src/main/java/org/libelektra/Elektra.java
Outdated
Show resolved
Hide resolved
src/bindings/jna/libelektra/src/main/java/org/libelektra/KDB.java
Outdated
Show resolved
Hide resolved
src/bindings/jna/libelektra/src/main/java/org/libelektra/plugin/PropertiesStorage.java
Show resolved
Hide resolved
src/bindings/swig/python/kdb.i
Outdated
return self.append(*list) | ||
|
||
def remove(self, name): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Problems with rebase?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think you use merge
instead of rebase
, and some weird issues appear on github.
I keep my fork's master identical to upstream, and always rebase from master:
git checkout master
git fetch upstream master # assuming you have upstream set already
# do not do this if you ever work on your master branch
git reset --hard upstream/master # this is fine, I never work on master and I only want it to mirror upstream
git checkout mybranch
git rebase master
During this process you might have to fix some conflicts manually.
Maybe there are improvements to this process, but I think we should probably put this somewhere in the docs.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I would also recommend, that the master
branch of your fork is kept identical to that of the main repo.
I have set up my remotes like this (output of git remote -vv
):
origin git@github.com:kodebach/libelektra.git (fetch)
origin git@github.com:kodebach/libelektra.git (push)
upstream git@github.com:ElektraInitiative/libelektra (fetch)
upstream git@github.com:ElektraInitiative/libelektra (push)
To update the master
branch I do:
git fetch upstream master:master
git push origin master:master
This works without switching to the master
branch first. After that you can do git rebase master
to rebase the current branch.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@kodebach can you copy what you wrote here to doc/GIT.md?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sure, but do we really need a git tutorial? Doesn't Elektra have enough barely maintained docs already?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The ksRewind problem is reproducible. This needs to be fixed before merging this. |
Aren't you using Gradle Wrapper? There should be no need to install gradle through homebrew. AFAIK the point of Gradle Wrapper is that it downloads a fixed version and uses that without the need to have a system-wide gradle install.
This is not an error. The The fact that the errors alternate, seems to suggest that libelektra/src/bindings/jna/libelektra5j/src/test/java/org/libelektra/GOptsTest.java Line 51 in 97d945d
No further code should be executed by
This suggests that it happens during There is one more possibility. I noticed that |
@kodebach thank you for insights!
Gradle wrapper would download gradle each time a fresh container is used for building, therefore we decided to include the gradle binaries with the images.
That makes sense!
I concur with this conclusion, but the thing that bugs me is that the problem does not seem to occur on
I am trying to narrow it down a little further right now. @mpranj @kodebach If i continue to fail getting to the root of the problem, we would have to think what we should do about today's release. Disable GOptsTest and list it as known issue? |
… be available before (if I'm interpreting the output correctly)
Then you should probably make sure that the system-wide install of Gradle is the same version as the one configured via Gradle Wrapper. That means using e.g.
The memory bug might be present on Just because there is a memory bug does not necessarily mean there will be a segfault. That is danger of using a language with manual memory management.
I can't make a decision about the release, but as long as we don't actually know the cause of the segfault, we probably shouldn't make a release. |
I now can reproduce it locally with the defualt resolver set as mentioned above. Interestingly now at I will now debug the JNA GOptsTests to narrow down what exact call to the native library triggers the problem down the line.... |
It seems just removing the |
I think you should still add it, in case the pre-installed Gradle version changes.
Cirrus CI uses Anka for it's macOS VMs and their configs can be found here: https://github.com/cirruslabs/osx-images See also: https://cirrus-ci.org/guide/macOS/ |
….8.3 as recommended by @kodebach
this results in |
5172ac5
to
405e2b1
Compare
I don't use Homebrew, possibly my quick Google search didn't give the correct way to install a fixed version.
A minimum version isn't enough I think. Fixing a major version might be enough, but there are definitely Gradle releases with breaking changes (i.e. not backwards compatible) from time to time. |
Final analysis from my side follows Steps to reproduce
The last call to the native library is triggered by I've included some JVM crash dumps jvm_crash_dumps.zip I am afraid, at the moment, i cannot be of further help regarding this issue. |
… for macOS_gcc10.yml" This reverts commit 3a655a4.
I have opened ticket #3770 to track this issue. |
Thank you all for working hard on this! ❤️ Please ping me if anything changes in the mean time. (Fyi, I'll copy this PR to a different branch so I can test independently.) |
I was able to get rid of the segfault by fixing the test exclusion mechanism. The root cause was that the GOpts Tests were executed on a system where the EDIT: I also think this has noting to do with ksRewind at all. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Great work, nice to see such a refresh to the Java bindings.
|
||
Release versions of the Elektra JNA bindings are published to Maven Central with group id `org.libelektra`, artefact id `libelektra` and the same version as Elektra. | ||
|
||
If you want to depend on a modified binding or just want to install it to your local maven repository, change your current working directory to the JNA binding folder `/src/bindings/jna`. Either use the bundled Gradle wrapper (`./gradlew` for Unix style OS or `./gradlew.bat` for windows) or ensure a recent Gradle version is available on your system and execute: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is it recommended that we keep both the wrapper and also installed versions of gradle, or should we get rid of one of the two now?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Developers might have different Gradle versions installed. The wrapper is the recommended way to get consistent Gradle versions AFAIK.
Within CI and Docker it might be problematic (i.e. a waste of time) to let Gradle Wrapper download the Gradle binaries every time. If this can be avoided (e.g. via caching or multi-stage Docker builds), I think it would be preferable to use Gradle Wrapper everywhere. If this is not possible, then we should probably list all the places where Gradle Version is defined in a central location.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
PS. for in the README I would only recommend using Gradle Wrapper, since that gives a consistent environment
Basics
These points need to be fulfilled for every PR:
(added as entry in
doc/news/_preparation_next_release.md
whichcontains
_(my name)_
)Please always add something to the release notes.
(first line should have
module: short statement
syntax)close #X
, are in the commit messages.doc/news/_preparation_next_release.md
scripts/dev/reformat-all
If you have any troubles fulfilling these criteria, please write
about the trouble as comment in the PR. We will help you.
But we cannot accept PRs that do not fulfill the basics.
Checklist
Check relevant points but please do not remove entries.
For docu fixes, spell checking, and similar none of these points below
need to be checked.
(not in the PR description)
Review
Reviewers will usually check the following:
Labels
If you are already Elektra developer:
say that everything is ready to be merged.