CI/CD Pipeline

[alibabashack]

We still need a working Continuous Integration and Delivery Pipeline. I think this is not trivial, because rxtx is neither a pure Java nor a pure C project. Since we use maven to build Java and C parts and even perform cross platform builds to problematic platforms (hallo apple!), I am not sure if there exists an off-the-shelf solution.

Dear CI/CD heads, Github Actions hackers, Jenkins wizards, we do need your input. Any idea what fits best for our scenario? It certainly would be nice if we can make this happen without own infrastructure.

[christianlupus]

Maybe I can help you here a bit. I will have a look tomorrow and set up a GH action to build the app. Mainly, #24 is required to be satisfied in order to have a clear idea of what needs to be done in CI/CD.

Also, the steps in the pipeline need to be specified. Should test suites be run automatically? Should there be a binary output of some sort?
In general, GH actions should make pretty much everything possible but the details might get messy. Therefore, we should know what we need exactly.

[alibabashack]

Wonderful! The build itself already works and is well defined from my perspective. #24 only targets the human readable state of the documentation. http://rxtx.github.io/rxtx/developers/build/instructions/release-build.html pretty much covers what needs to be done. We can build for all Linux and Windows machines from a single Ubuntu Linux. The only exception is the OSX build, because the relevant cross compiler is no longer maintained. So we would need to run the build of this particular binary on OSX. But the process already covers the build on different systems and the later assembly of the parts into the final jar.

For smoke tests we have hardware dependencies. So I don't think the cloud will help with this. In the long run we might need some kind of manual voting stages within the pipeline to check manual tests have been carried out on the relevant platforms. (Only for true release builds.)

And we do not have any unit tests for now. I'm not sure about the future, but I think we should make the build run first.

[alibabashack]

Hi @christianlupus ,
fyi: I started an effort to build using github actions on my private development branch https://github.com/alibabashack/rxtx/tree/development It builds now for all supported platforms. Due to some maven issue the resulting rxtxSerial.jar does not include some native libs. As soon as I have this fixed, I will cleanup and create a merge request.

Feedback welcome!

[christianlupus]

@alibabashack Sorry, I was busy the whole week with my day job. But you seem to have successfully implemented a CI using GH actions. Looks good to me.
I personally would have merged the apt calls but this is only a personal preference. So do not worry about it.

Another option that might be a good idea would be to extract the main tasks from the work files into their own actions. That would allow e.g. to make the java version configurable. Once we have a new version you just have to change that reference. And as a side effect, you could build for different java versions. I am not very comfortable with maven, so I am unsure, if this is needed.

[alibabashack]

You are right about the apt calls, I had the same thing in mind. It would be faster to have a single run and apt would make clear what's missing in case of an error.

I don't understand your idea about multiple actions. Could you give an example when this is an advantage, please? Why would I want builds with different java versions? Or are you referring to the rxtx lib version?

To be honest, I don't know maven at all, apart from what's been done here. It seemed a good idea to me eight years ago to migrate the build to maven, because I had the impression it was the natural Java way to go. I personally dislike the xml and declarative approach. Hacking the build setup always feels slow and painful. And at least for the native C part it does not integrate well with the usual tools. I was forced to mirror the setup as a CMake config in my private repository in order to integrate with my IDE setup. But I think the most important goal is to make Java programmers feel comfortable. I think most users don't usually deal with C or C++ and tools like Make, CMake or autotools. I remember this had been my perspective before I went into the embedded world. So do you think Java or other JVM language developers would still choose maven for the build? Or do they only care about maven for distribution/dependency management?

[christianlupus]

I don't understand your idea about multiple actions. Could you give an example when this is an advantage, please? Why would I want builds with different java versions? Or are you referring to the rxtx lib version?

No, I was referring to the java version. I am unsure if there is a benefit to have different versions. In other libraries (with other programming languages) it might be required to simplify the testing and the like.

To be honest, I don't know maven at all, apart from what's been done here. It seemed a good idea to me eight years ago to migrate the build to maven, because I had the impression it was the natural Java way to go. I personally dislike the xml and declarative approach. Hacking the build setup always feels slow and painful. And at least for the native C part it does not integrate well with the usual tools. I was forced to mirror the setup as a CMake config in my private repository in order to integrate with my IDE setup. But I think the most important goal is to make Java programmers feel comfortable. I think most users don't usually deal with C or C++ and tools like Make, CMake or autotools. I remember this had been my perspective before I went into the embedded world. So do you think Java or other JVM language developers would still choose maven for the build? Or do they only care about maven for distribution/dependency management?

Honestly, I am no too much into Java anymore. I learned Java as my first language and I am still confident but never went to the point where depency management came into play. I know that maven is the standard tool for the java world. So, I am 100% with you (dislike of XML, multi-language tool problems).

Currently, I am heavily using python and PHP/JS and at least for PHP it is useful to test against different dependencies (aka PHP versions). Therefore, I used to pack the main tasks into a separate action with an input that allows individual parameters. It was just an idea out of curiosity if it would be needed.

[alibabashack]

No, I was referring to the java version. I am unsure if there is a benefit to have different versions. In other libraries (with other programming languages) it might be required to simplify the testing and the like.

As far as I understand, the JVM bytecode should be stable or newer machines would be at least backward compatible to some extend. So the only issue, where the Java version might matter in the future, is API or language feature deprecation. So I don't see a reason here to make CI/CD work with multiple JDKs at this time. For now it is a huge improvement to have a build running and being able to distribute binaries again.

I have submitted pull request #32 with a working pipeline. It includes some maven fixes as described in the commits. I also merged the apt install calls to a single invocation as discussed.

[alibabashack]

merged #32