Now that gtk2 is dead can you start compiling arm32/64 OpenCPN with gtk3? #2138
Comments
|
I still have to catch all of Sean's plugins up with the previous cycle.... I cannot keep up with these changes. Can this change get included with the current testplugin status, before I start updating? |
|
There is already an official Debian package supporting arm which is gtk3 only. The problem is the plugins, and building for new platforms is a pure manpower problem. I'm sure Dave would happily accept a PR against the plugins project with new, gtk3 builds for arm. |
|
Gtk4 is out too. Might be better skip gtk3 and go straight gtk4. |
|
It keeps moving. No problem. I can wait to update all Sean's plugins again, but it will be some time period maybe a month before everything gets figured out. |
|
https://developer.gnome.org/gtk4/unstable/gtk-migrating-3-to-4.html
Migration guide.
…Sent from my iPhone
On Feb 17, 2021, at 5:29 PM, Rick Gleason ***@***.***> wrote:
It keeps moving. No problem. I can wait to update all Sean's plugins again, but it will be some time period maybe a month before everything gets figured out.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub, or unsubscribe.
|
|
Oh really? Look at that huge list. Maybe we should get real about this? Just my opinion. |
|
@rgleason the above is completely irrelevant for you, ignore. |
|
@mgrouch |
"rPI" is a bit ambiguous in this context. I suppose you mean raspbian, which is used by most users (these days, standard Debian runs on rPI, and so does Ubuntu. But that's another story) Current Raspbian is based on Buster which certainly has wxWidgets-gtk3 packaged since long: EDIT s/gtk3/wxWidgets-gtk3/ |
|
The other part is that we don't know where the rPI ecosystem is heading. So far, Raspbian has had a firm grip on the market, due to historical reasons, graphics driver etc. However, since the standard linux kernel now supports rPI there is now a different situation. Here is also a move towards a full 64-bit solution (rPI 4 comes with 8 Gb), and we see this as attempts by @mgrouch and others to port OpenCPN to 64 bit Arm using standard Debian. I see the signs that we might need to start to build 64-bit plugins on Arm for some Linux distro. Hopefully just one, with Debian Bullseye as most likely alternative (?) |
|
Pavel, you make a point, but some of these changes shuffle down to plugin builds, particularly gtk and wiring inevitably has to change for each plugin. |
|
|
|
Indeed. But we will still need to move to gtk3 since any upcoming Raspbian version based on Bullseye (probably next) will drop support for gtk2. So we have no choice, this change is inevitable. Since Buster has been around for around 18 months it might possibly make sense to declare that next OpenCPN release on Arm will be using gtk3, thus creating an ABI break. The 64-bit issue is of the same kind, but more diffuse. Which will be the dominating 64-bit OS on rPI? Raspbian (64bit version is in beta), Debian, Ubuntu, ... We will have to make a decision, but it might be too early. One way would of course be to enable flatpak aarch64 builds, and build plugins for this. Perhaps the most reasonable 64-bit option today. Using flatpak is the way to avoid fragmentation, for sure. |
|
Or might be have flatpack builds for supported platforms and use these flatpacks as the source to create native packages .deb .rpm, etc. That should be fast as no compilation is involved.
…Sent from my iPhone
On Feb 18, 2021, at 11:25 AM, leamas ***@***.***> wrote:
Indeed. But we will still need to move to gtk3 since any upcoming Raspbian version based on Bullseye (probably next) will drop support for gtk2. So we have no choice, this change is inevitable. Since Buster has been around for around 18 months it might possibly make sense to declare that next OpenCPN release on Arm will be using gtk3, thus creating an ABI break.
The 64-bit issue is of the same kind, but more diffuse. Which will be the dominating 64-bit OS on rPI? Raspbian (64bit version is in beta), Debian, Ubuntu, ... We will have to make a decision, but it might be too early.
One way would of course be to enable flatpak aarch64 builds, and build plugins for this. Perhaps the most reasonable 64-bit option today. Using flatpak is the way to avoid fragmentation, for sure.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or unsubscribe.
|
Nope, wont work. Neither technical (different runtime env) nor administrative. The reason to use flatpak would be to have one single package for all 64-bit aarch64 machines. If we still wan't to compile for different os'es we are back on square one. BTW: It's spelled flatpak :) |
|
I think circleCI is not a good place to build anything arm based. |
|
Problem is that Travis is not free for open-source projects anymore, so it's kind of dead end. I have been looking at Shippable as an alternative, but nothing really done -- help appreciated! EDIT: If going this route, building flatpak plugins should IMHO be the first priority. We could enable the aarch64 flathub builds at any point, it's basically just a line of configuration. And given that the 64-bit Arm/Raspberry world seems to be more fragmented than armhf, flatpak should be the first option. |
|
Launchpad supports daily automatic builds. Launchpad builds for arm are quite fast too.
https://help.launchpad.net/Packaging/SourceBuilds
…Sent from my iPhone
On Feb 20, 2021, at 4:19 AM, leamas ***@***.***> wrote:
Problem is that Travis is not free for open-source projects anymore, so it's kind of dead end. I have been looking at Shippable as an alternative, but nothing really done -- help appreciated!
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or unsubscribe.
|
|
But it doesn't work. Launchpad builds .deb packages, we need to build arbitrary tarballs. If we could find a way to use the Debian/Launchpad/COPR/OBS whatever build chains it would be really interesting. I have thought about it,but inconclusive. |
|
.deb packages can be converted somewhere else into tarballs
…Sent from my iPhone
On Feb 20, 2021, at 10:14 AM, leamas ***@***.***> wrote:
But it doesn't work. Launchpad builds .deb packages, we need to build arbitrary tarballs.
If we could find a way to use the Debian/Launchpad/COPR/OBS whatever build chains it would be really interesting. I have thought about it,but inconclusive.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or unsubscribe.
|
|
yes... but that is a complicated way, creatiung a debian packaging for each and every plugin, add the tarball generation to that, post-build jobs which install/downloads the debs, stores the tarball on something like cloudsmith, generates the metadata, builds the catalog PR. It is probably doable, but a complicated mess compared to the current solution where we just builds tarballs, uploads and generates metadata on the fly. My initial thought is that it's just not worth it. |
|
But launchpad can build on so many platforms and for so many Debian flavors. And it’s fast for arm builds.
One can re-use Debian packaging files (most of O plugins had it) in the LaunchPad recipe. Daily recipe will trigger builds on changes in upstream. Once this setup it should be low maintenance.
…Sent from my iPhone
On Feb 20, 2021, at 11:17 AM, leamas ***@***.***> wrote:
yes... but that is a complicated way, creatiung a debian packaging for each and every plugin, add the tarball generation to that, post-build jobs which install/downloads the debs, stores the tarball on something like cloudsmith, generates the metadata, builds the catalog PR.
It is probably doable, but a complicated mess compared to the current solution where we just builds tarballs, uploads and generates metadata on the fly. My initial thought is that it's just not worth it.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or unsubscribe.
|
|
Well, as they say: patches welcome. When it comes to Linux, the fragmentation is about Debian, so using the Ubuntu build farms makes sense (exception: non-free plugins like oesenc). Note that launchpad does not build for Debian, only Ubuntu. So there is one catch. |
|
I’ve played a bit with daily builds recipes on launchpad. It not that difficult to set up. The recipe is just 4 lines. the git repository with OpenCPN needs to be imported into launchpad (I did it as git but there is also bazaar option)
Then another repository with just Debian packaging files needs to be imported into launchpad.
Launchpad will perform automatic syncs every 5 hours from those 2 repositories on GitHub.
in the recipe you will have one
merge statement
And another nest-part to fetch debian directory.
and then after importing the recipe into launchpad you can have daily builds for all Ubuntu flavors on all supported platforms.
the maintenance will be only maintaining proper Debian packaging project for OpenCPN.
in launchpad you could check daily logs. If builds are successful it will automatically put results into daily PPA.
…Sent from my iPhone
On Feb 20, 2021, at 12:13 PM, leamas ***@***.***> wrote:
Well, as they say: patches welcome. When it comes to Linux, the fragmentation is about Debian, so using the Ubuntu build farms makes sense (exception: non-free plugins like oesenc).
Note that launchpad does not build for Debian, only Ubuntu. So there is one catch.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or unsubscribe.
|
|
Yes, I'm not not that surprised that these parts works without too much hassle. Now, the remaining work on this path if I get it right, out of the top of my head:
Everything here is certainly doable. The upside is the well-maintained Ubuntu build farm, with a more or less complete Ubuntu versions / hardware matrix. The downside is that this will be a complicated setup which also creates long delays between pushing a commit and checking the result. Also, it doesn't really apply to Flatpak, Debian or non-free plugins like oesenc. And, not least, it would force plugin developers to sign the Ubuntu Contributor Agreement -- not all want to do this. Bottom line: I see the possibilities but I'm not convinced the result is worth the costs. That is certainly not to say I'm certain. BTW: this is indeed interesting. |
|
Just for the sake of discussion: I don't really believe in the nightly jobs approach. It's probably better to stick to the traditional PPA workflow where developers creates a source package and uploads it. Developers knows when they want to build (typically not every night), and when it's time they don't want to wait for a nightly job. But, this is a detail in the bigger picture. |
|
The correct approach is to have continuous build (build triggered by every commit). This one is daily build which doesn’t get triggered if there was no changes (afaik). About 5 hours polling interval which is not that bad.
Developers uploading into launchpad is too much effort for them. There are too many steps. With this all they need it to push into GitHub.
…Sent from my iPhone
On Feb 21, 2021, at 3:17 PM, leamas ***@***.***> wrote:
Just for the sake of discussion: I don't really believe in the nightly jobs approach. It's probably better to stick to the traditional PPA workflow where developers creates a source package and uploads it. Developer knows when they want to build (typically not every night), and when it's time they don't want to wait for a nightly job. But, this is a detail in the bigger picture.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or unsubscribe.
|
|
Perhaps, perhaps not. But these are details. The bigger question is if the upside motivates the costs. I'm not convinced... |
|
With uploading to launch pad you have to repeat-yourself. Which violates principle of do-not-repeat-yourself. You have to upload a package for each distribution with manually edited Debian changes file.
Here your Debian packaging work completes after a commit.
And recipe does it for all Ubuntu distros on all platforms.
Developer I do not think they need to sign Ubuntu agreement. Only the person who sets up recipe.
…Sent from my iPhone
On Feb 21, 2021, at 4:16 PM, leamas ***@***.***> wrote:
Perhaps, perhaps not. But these are details. The bigger question is if the upside motivates the costs. I'm not convinced...
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or unsubscribe.
|
|
@mgrouch If you want to setup launchpad with debian plugins, perhaps you can create a script to use these. I might be useful to get a free opensource Cloudsmith account using your git login. |
|
Just my 2 cents, after I have created hundreds of PPA builds for both OpenCPN and many plugins over a period of several years:
|
|
Pavel, all your great efforts were hugely appreciated by many unknowing users! |
|
You haven’t even tried making recipe on launchpad. I agree that manual launchpad process if flawed. I’m talking about automatic one
…Sent from my iPhone
On Feb 21, 2021, at 5:34 PM, Rick Gleason ***@***.***> wrote:
Pavel, all your great efforts were hugely appreciated by many unknowing users!
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or unsubscribe.
|
Oh yes, I did and it even did somehow work sometimes. Complete joke compared to any other CI system out there though. But don't take me wrong, when I see a pull request implementing what you are suggesting and does actually work, I will of course give you big thumbs up and hit that merge button myself. |
|
Pull request not going to create a project on launchpad under OpenCPN. It’s not going to import GitHub repositories into launchpad either. recipe itself is just 4 lines. Once you set it up in launchpad (Right now it’s just ppa) I can make a pull request with those 4 lines if that helps.
…Sent from my iPhone
On Feb 21, 2021, at 5:53 PM, Pavel Kalian ***@***.***> wrote:
You haven’t even tried making recipe on launchpad. I agree that manual launchpad process if flawed. I’m talking about automatic one
Oh yes, I did and it even did somehow work sometimes. Complete joke compared to any other CI system out there though.
But don't take me wrong, when I see a pull request implementing what you are suggesting and does actually work, I will of course give you big thumbs up and hit that merge button myself.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or unsubscribe.
|
|
I'm personally afraid this is not how it can work. What on Earth is prohibiting you from demonstrating the (whole, doing something that actually solves a problem of course, not 4 lines of recipe "code") workflow on your clone as everybody else does? But of course just show me the 4 lines if you are convinced they are enough to actually do all of it. |
|
You could create a script that downloads the deb files to wherever, similar too this one for windows These are just examples. |
|
mgrouch "The correct approach is to have continuous build (build triggered by every commit)." Not so sure about that. For example I do not build locally for every environment, just windows. Thus other builds may fail and have to be worked on. You may not want Launchpad to process everything pushed to git, all the time. |
Here is the demonstration. I've set up recipe in launchpad now. https://code.launchpad.net/~bbn-projects/+recipe/bareboat-necessities-projects-daily The recipe (to build OpenCPN trunk into .deb packages for all current Ubuntu versions on all platforms): git-build-recipe format 0.4 deb-version {debupstream}-0~{revtime}lp:~bbn-projects/bareboat-necessities-projects/+git/OpenCPN master Where and lp:bareboat-necessities-projects debian/sid Only debian directory is taken from it The builds are failing now for the reasons of dependencies not specified correctly in So this gives leamas a view of why the builds are failing and chance to correct debian packaging. If there is some other code in git where I can pick up debian packaging files for opencpn I can try those as well. |
|
For example: Bionic build Focal build Groovy build (same as previous) Hirsute (same issue): |
|
So now all we need is a few, I suppose trivial, pull requests fixing your findings. And then what? Again, this is not saying that what you suggest is total crap, I just want to see it aligned with reality we live in. And again, I am much more interested in seeing pull requests fixing issues than list of "this is broken, fix it somehow" items. |
|
To be more of a nitpicker, the "bugs" you are pointing out here all refer to https://gitlab.com/leamas/opencpn which is not even part of this project but a single Linux distribution, not being Ubuntu, specific packaging related repository. |
|
If you are interested in how the PPA was maintained when it was done by me, you may have a look at https://github.com/nohal/launchpad The "debian directory" of course can't be taken from there and applied to absolutely any DEB based distro and expected to be usable neither. |
|
So there is need for several branches for Debian packaging. These can be a separate project without OpenCPN code base. And one recipe per one of those branches.
The end product which is for Debian based distributions is .deb package should be testable after each commit into OpenCPN git ideally. To be testable it at least need to exist (be built). Daily build doesn’t solve each commit validation but at least it solves issue of validation of “buildability” on all target Debian platforms*distros of one day of work done in OpenCPN.
…Sent from my iPhone
On Feb 21, 2021, at 10:23 PM, Pavel Kalian ***@***.***> wrote:
To be more of a nitpicker, the "bugs" you are pointing out here all refer to https://gitlab.com/leamas/opencpn which is not even part of this project but a single Linux distribution, not being Ubuntu, specific packaging related repository.
I am more than convinced that it is impossible to both make the "debian directory" acceptable for Debian (sure not more than one specific release of it) and many Ubuntu releases at the same time, so problems like what you outline are as far as I'm concerned absolutely expected and probably not even resolvable by a trivial recipe I see.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or unsubscribe.
|
|
Where can I find most appropriate Debian directory with which OpenCPN master branch would build right now on most distros?
I’d like to set it up with recipes that it builds on all platforms/distros in launchpad.
…Sent from my iPhone
On Feb 21, 2021, at 10:37 PM, Pavel Kalian ***@***.***> wrote:
If you are interested how the PPA was maintained when it was done by me, you may have a look at https://github.com/nohal/launchpad
What I can tell about it is it sure was not perfect and I would have done it differently doing it now with the current CI tools available, but the amount of work it generated was still minutes per month and thus completely acceptable.
The "debian directory" of course can't be taken from there and applied to absolutely any DEB based distro and expected to be usable neither.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or unsubscribe.
|
|
That Debian packaging needs to refer to latest released wxWidgets which is 3.0 (am I right?) and wxWidgets need to be setup to use latest available gtk3. There are some third party libraries which are copied into OpenCPN codebase (I guess for a reason) and it needs to use those to compile statically instead of pulling other versions of those. Is that how OpenCPN is built?
…Sent from my iPhone
On Feb 21, 2021, at 10:46 PM, Mikhail Grushinskiy ***@***.***> wrote:
Where can I find most appropriate Debian directory with which OpenCPN master branch would build right now on most distros?
I’d like to set it up with recipes that it builds on all platforms/distros in launchpad.
Sent from my iPhone
>> On Feb 21, 2021, at 10:37 PM, Pavel Kalian ***@***.***> wrote:
>>
>
> If you are interested how the PPA was maintained when it was done by me, you may have a look at https://github.com/nohal/launchpad
> What I can tell about it is it sure was not perfect and I would have done it differently doing it now with the current CI tools available, but the amount of work it generated was still minutes per month and thus completely acceptable.
>
> The "debian directory" of course can't be taken from there and applied to absolutely any DEB based distro and expected to be usable neither.
>
> —
> You are receiving this because you were mentioned.
> Reply to this email directly, view it on GitHub, or unsubscribe.
|
|
There are at least two completely different beasts:
For OpenCPN, we must be VERY conservative about any ABI changes as they will trigger the need to rebuild, repackage and redistribute all the plugins. And we have absolutley no control over the plugins. We simply have no "take this, build it, it works" you keep asking for. We have "You change anything, you might break a lot of stuff". I don't know what "libraries copied into OpenCPN codebase" relevant to Linux you talk about - as far as I can recall everything we can expect to be packaged on Linux we detect and link dynamicaly if it is available. We have absolutely no control over which wxWidgets versions are available on which Linux distribution and it's version and what version of Gtk they are built against. And we do not care, if there is a problem that is reported, widespread and severe enough, we try to work it around. The workaround is NEVER providing our own wxWidgets or Gtk build. |
|
Looking at this part in CMake.txt
If you find gtk2 even if there is gtk3 on the system
you would link against gtk2. It doesn’t look
right to me, given gtk2 is basically obsolete
and you do support gtk3.
if (NOT WIN32 AND NOT APPLE AND NOT QT_ANDROID)
find_library(GTK2U NAMES wx_gtk2u_core)
if (GTK2U)
option(OCPN_FORCE_GTK3 "Force the build to use GTK3 [OFF]" "OFF")
else ()
option(OCPN_FORCE_GTK3 "Force the build to use GTK3 [ON]" "ON")
endif ()
endif ()
…Sent from my iPhone
On Feb 21, 2021, at 11:14 PM, Pavel Kalian ***@***.***> wrote:
There are at least two completely different beasts:
DEB acceptable in Debian
DEB working "everywhere" we want to (This is a unicorn that at times does not even exist), which is most likely totally unacceptable for Debian maintainers at any moment
For OpenCPN, we must be VERY conservative about any ABI changes as they will trigger the need to rebuild, repackage and redistribute all the plugins. And we have absolutley no control over the plugins.
We simply have no "take this, build it, it works" you keep asking for. We have "You change anything, you might break a lot of stuff". I don't know what "libraries copied into OpenCPN codebase" relevant to Linux you talk about - as far as I can recall everything we can expect to be packaged on Linux we detect and link dynamicaly if it is available.
On other platforms like Windows, macOS and Android, all of which we do support and have more users on them than on Linux, there are libraries that we must provide, build and link ourselves as they are not otherwise available there.
We have absolutely no control over which wxWidgets versions are available on which Linux distribution and it's version and what version of Gtk they are built against. And we do not care, if there is a problem that is reported, widespread and severe enough, we try to work it around. The workaround is NEVER providing our own wxWidgets or Gtk build.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or unsubscribe.
|
|
What does this break and on what platform? The purpose of this particular build option (and we just set a default for it here, still can be modified by the user running the build) was to work around a (really existing) problem on Linux systems where both wxWidgets linked against Gtk2 and Gtk3 could be present based on user choice and the result being a totally unpredictable mess. I think we could probably already remove it nowadays as that mess is not present in any current distro version we care about. It never ever disabled support for Gtk3 if it was present on the system, less when it was the only Gtk version present. To be honest I would much prefer to talk about problems that do exist and need to be solved. |
|
hm... here are many open issues, at least from my point of view:
My personal view is that building main opencpn is not really an issue. We have a well working CI setup which builds each and every change. And from the upcoming Debian/Ubuntu releases OpenCPN will be built for all Debian [1] and Ubuntu [2] platforms. The bottleneck is the need for each new platform to maintain plugins. Today we use use CI setups, mostly on CircleCI for this. The idea to use Launchpad could fit into this puzzle, by building many but not all plugins for most Ubuntu version/hardware platform combinations. To repeat myself, this would benefit from the well maintained Ubuntu build farm but with some problems like overall complexity, non-free plugins (oesenc etc), Debian (doesn't work) and developer ideology + paperwork (Ubuntu Contribute Agreement, registered gpg keys etc.). In the end, the upsides might not justify the costs. OTOH, like @nohal says: a PR which makes it possible for a plugin like shipdriver to use the Launchpad infrastructure to build for Ubuntu platforms would certainly be interesting. |
|
But then again, the core issue here is the Linux fragmentation. When we now are about to enter the aarch64 Arm world, the most important piece is, again, to enable the main OpenCPN Flatpak aarch 64 build (simple) and add plugin build chains for it. This would ensure we have a working solution on all aarch64 platforms, and actually fight the fragmentation instead of just accepting it. |
We don't care about GTK flavors. We care about which GTK version does the system wxWidgets libraries use. And wxWidgets supports both GTK2 and GTK+ to coexist on the same platform. The builder has to manually run wx-config to set the desired toolkit (gtk2 or gtk3). So far, CMake does not do that part so it is a manual step done by the person building before running cmake. |
|
@transmitterdan: the rest of story seem to be that, for Debian wxWidgets:
These changes has trickled down to Ubuntu over time. Bionic supports gtk2 + gtk3, Focal is gtk3 only. I still think building main opencpn for more platforms besides just enabling for example Flatpak/aarch64 is a non-issue. EDIT: the reason for the logic in the patch (to default to gtk2 if it exists) was to not break existing setups which was created before wxWidgets/gtk3 was introduced. |
|
@leamas, You are right, but we should not drop GTK2 from CMakeLists.txt as that might break builds on some of these older platforms (e.g. buster, focal). The way CMakeLists.txt is now is a bit of a hack but it allows building on most of the Ubuntu flavors. Someday we can forget about gtk2. But not for perhaps a year or more. I also notice a lot more problems with gtk3 flavors of wxWidgets. I can create random crashes with gtk3 on focal amd64 and buster Arm builds, but gtk2 has proven remarkably stable. I hope gtk3 matures rapidly. Also, performance under gtk3 is noticeably slower on focal. |
|
@transmitterdan: there are lot's of applications out there on gtk3 which are neither slow nor unstable. So the missing point is probably the wxWidgets gtk3 adaptation. Re-tracking to the core issue here: there is no need to create new builds of main OpenCPN using gtk3. They will be upon us with Ubuntu Hirsute and Debian Bullseye for all possible platforms. Also, the focal builds already uses gtk3, it's the only option there. The problem is the Linux fragmentation and the corresponding need for multiple plugin builds. |
|
@transmitterdan: would of course be interesting if you could make some tests with the updated opencpn build in #2174, to see if we could have some faith in upcoming wx 3.2. No plugins, though. |
Gtk2 is EOL now. No new development and code is frozen.
And gtk2 version of OpenCPN on arm both 32/64 bit is buggy with touchscreen. Can we have official OpenCPN builds with gtk3 for arm?
Thanks
The text was updated successfully, but these errors were encountered: