ocpn-plugin.xsd inconsistent across alpha, beta and prod making it impossible to release new code

[jongough]

With each change to the plugin manager requirement the xsd's are inconsistent with each other, what has been discussed in the forums and in the issues here and change without notice. This has happened again! This make releasing any new code very difficult except for the person making the changes to plugin manger. I have just released a version of a plugin through alpha, beta and into prod. This eventually worked when the plugin manger and the xsd was finally made consistent across the levels. They have now changed again, without notice. Here are the versions:
Alpha ```
<xs:element name="target">
xs:simpleType
<xs:restriction base = "xs:string">
<xs:enumeration value = "all"/>
<xs:enumeration value = "android-arm64-v8a"/>
<xs:enumeration value = "android-armeabi-v7a"/>
<xs:enumeration value = "darwin"/>
<xs:enumeration value = "darwin-wx315"/>
<xs:enumeration value = "debian-armhf"/>
<xs:enumeration value = "debian-x86_64"/>
<xs:enumeration value = "flatpak-aarch64"/>
<xs:enumeration value = "flatpak-x86_64"/>
<xs:enumeration value = "mingw"/>
<xs:enumeration value = "mingw-x86_64"/>
<xs:enumeration value = "msvc"/>
<xs:enumeration value = "raspbian-armhf"/>
<xs:enumeration value = "ubuntu-arm64"/>
<xs:enumeration value = "ubuntu-gtk3-arm64"/>
<xs:enumeration value = "ubuntu-gtk3-x86_64"/>
<xs:enumeration value = "ubuntu-x86_64"/>
<xs:enumeration value = "android-armhf"/>
<xs:enumeration value = "android-arm64"/>
</xs:restriction>
</xs:simpleType>

Beta

<xs:element name="target">
xs:simpleType
<xs:restriction base = "xs:string">
<xs:enumeration value = "all"/>
<xs:enumeration value = "android-arm64-v8a"/>
<xs:enumeration value = "android-armeabi-v7a"/>
<xs:enumeration value = "darwin"/>
<xs:enumeration value = "darwin-wx315"/>
<xs:enumeration value = "debian-armhf"/>
<xs:enumeration value = "debian-x86_64"/>
<xs:enumeration value = "flatpak-aarch64"/>
<xs:enumeration value = "flatpak-x86_64"/>
<xs:enumeration value = "mingw"/>
<xs:enumeration value = "mingw-x86_64"/>
<xs:enumeration value = "msvc"/>
<xs:enumeration value = "raspbian-armhf"/>
<xs:enumeration value = "ubuntu-armhf"/>
<xs:enumeration value = "ubuntu-gtk3-armhf"/>
<xs:enumeration value = "ubuntu-gtk3-x86_64"/>
<xs:enumeration value = "ubuntu-x86_64"/>
<xs:enumeration value = "android-armhf"/>
<xs:enumeration value = "android-arm64"/>
</xs:restriction>
</xs:simpleType>
</xs:element>

Prod

<xs:element name="target">
xs:simpleType
<xs:restriction base = "xs:string">
<xs:enumeration value = "all"/>
<xs:enumeration value = "android-arm64-v8a"/>
<xs:enumeration value = "android-armeabi-v7a"/>
<xs:enumeration value = "darwin"/>
<xs:enumeration value = "darwin-wx315"/>
<xs:enumeration value = "debian-armhf"/>
<xs:enumeration value = "debian-x86_64"/>
<xs:enumeration value = "flatpak-aarch64"/>
<xs:enumeration value = "flatpak-x86_64"/>
<xs:enumeration value = "mingw"/>
<xs:enumeration value = "mingw-x86_64"/>
<xs:enumeration value = "msvc"/>
<xs:enumeration value = "raspbian-armhf"/>
<xs:enumeration value = "ubuntu-arm64"/>
<xs:enumeration value = "ubuntu-gtk3-arm64"/>
<xs:enumeration value = "ubuntu-gtk3-x86_64"/>
<xs:enumeration value = "ubuntu-x86_64"/>
<xs:enumeration value = "android-armhf"/>
<xs:enumeration value = "android-arm64"/>
</xs:restriction>
</xs:simpleType>
</xs:element>

the 'ubuntu-armhf' has now become 'ubuntu-arm64', why?

[bdbcat]

Jon...
If you look at the history of this file (ocpn-plugin.xsd), in all three branches, you will see that nothing has recently changed.
Looks to me as though this is an error of omission in the master branch, long standing. Not too surprising, since plugins actually using the ubuntu-armxx targets are very new, only a few weeks old. And it turns out that yours is the first to go to Production. So you found the error first.
Easily fixed. Done by 6aabaae
Dave

[jongough]

Dave,
This is a problem that keeps occurring as there is only 'one' correct xsd. There is no backwards compatibility built in which means if it changes to allow new environments it normally invalidates others. With the non-standard flow of xsd's (seems to be beta-prod->alpha) and the moment (although alpha is/was the first this time) it makes it very difficult to generate what plugin manager wants.

Can the process be updated such that new items are put in alpha first then migrated through beta into prod? If there is a need to put it into prod quickly then both alpha and beta need to be updated at that time.

I have just tried with the new prod and got:

etadata/ocpn_draw_pi-1.8.13.0-ubuntu-armhf-20.04.xml:17: element target: Schemas validity error : Element 'target': [facet 'enumeration'] The value 'ubuntu-gtk3-armhf' is not an element of the set {'all', 'android-arm64-v8a', 'android-armeabi-v7a', 'darwin', 'darwin-wx315', 'debian-armhf', 'debian-x86_64', 'flatpak-aarch64', 'flatpak-x86_64', 'mingw', 'mingw-x86_64', 'msvc', 'raspbian-armhf', 'ubuntu-armhf', 'ubuntu-arm64', 'ubuntu-gtk3-arm64', 'ubuntu-gtk3-x86_64', 'ubuntu-x86_64', 'android-armhf', 'android-arm64'}.

So is ubuntu arm gtk3 only supported on arm64 and not armhf for ubuntu 20.04?

[bdbcat]

"So is ubuntu arm gtk3 only supported on arm64 and not armhf for ubuntu 20.04?"
Correct.
Also, please know that native arm64 plugins are not "officially" supported in any case, except for Android and flatpak, as special (non-ubuntu) cases. So, at his point, there is no need to build any native arm64 plugins at all.

On the XSD files: Jon, you are leading the curve here by using, in sequence, all three branches. Alpha is not much used by others, in general. Beta neither, except for carefully selected testers. I've instructed Rick to publish to master directly, so that we get immediate feedback. Anyway, as a result of using all 3 branches, you might expect to be the first to find some anomalies in the build tooling. When you find them, surface them as you have, and they get fixed. That is the process. Each step is an improvement. No drama.

[jongough]

So if ubuntu-gtk3-armhf is not allowed why is debian-armhf which is built with gtk3 allowed? The ubuntu version only has gtk3 in it because plugin manager requires that it be identified as such.

[bdbcat]

"So is ubuntu arm gtk3 only supported on arm64 and not armhf for ubuntu 20.04?"
My answer was not correct. Sorry.

Looks like another bug. "ubuntu-gtk3-armhf" is the target for Raspbian Bullseye, and it uses gtk3.
On the other hand, "ubuntu-armhf" is not used anywhere, AFAICT, so should be removed.
I'll fix it.

[bdbcat]

Committed revised ocpn-plugin.xsd to all three branches, exactly the same for all.
Thanks for the catch.
Dave

[jongough]

Just checked and the 'debian-armhf' targets are not in any of the xsd's. I know these are the latest targets as that is what I am trying to get frontend2 to deliver for the other plugins. As the xsd's and plugin manager are tightly coupled I am not game to make any changes to the xsd as the plugin manager is opaque to me.

[bdbcat]

Jon...
You are way ahead of the curve here. Nothing wrong about that. But I do not think that we are really ready for debian-armhf plugins in OCPN core, yet. I am not at all sure that O56 will be able to load "debian-armhf" plugins on any platform we support, for example rPI Buster or Bullseye. This is to be verified.
Meanwhile, we are ready for "ubuntu-armhf" plugins now, and that is our next stop on the roadmap.

A point of information:
XSD's and plugin manager are not as tightly coupled as you may think. The xsd's are used exclusively as a sanity-check of metadata before allowing a commit to the plugins repo. XSD's were originally important while the plugin CI system was being developed, when metadata errors were common. They are less important now that the various CI Frontends are beginning to stabilize.

Importantly, in no way does existence of a target in the XSD imply that OCPN56 promises the capability of loading such a plugin. Indeed, OCPN-runtime has no knowledge of the XSD files in the plugins repo.

So, feel free to build and test for any target you wish. Post a PR to add targets to the XSD, if necessary, for publication. What really happens in OCPN at runtime is another thing entirely.

[jongough]

I only built the debian-arm* process as I had an issue raised against testplugin to do so jongough/testplugin_pi#238. Testplugin (frontend2) now builds for this but does need a test environment that will work. It sounds like this is still a way off.

[bdbcat]

Please see:
jongough/testplugin_pi#238 (comment)

[jongough]

There is still a problem with inconsistencies. The alpha version has "darwin-wx32", the beta version has "darwin-wx321" and prod version has "darwin-wx32". Can there please be consistency between git branches? The current situation makes it impossible to build for alpha, move to beta and then to prod.

[bdbcat]

Done. Thanks for the catch.

[jongough]

It would appear using a standard process for changes starting in alpha and being migrated through to prod would alleviate these issues. Emergency fixes would have their own process to get them into prod, but part of that would be to retrofit them through alpha and beta as soon as reasonable.

Thanks

[rgleason]

So with these changes to the plugin gatekeeper script (for examples see below) , I expect it is unlikely that we will be able to recompile O5.6.2 plugins and push them to opencpn/plugins. I suppose you could have a gatekeeper script that allowed both O5.6.2 and O5.7.1 but this is some work. Also the ci scripts for O5.7.1 would have to be saved.

So if the O5.6.2 plugins files are lost on Cloudsmith for some reason, they are history. The only way to keep them intact is to back them up somewhere and allow users to "Import Plugin" from some backup location.

So this is a long way of saying that it is highly unlikely that we will be able to update the O 5.6.2 plugins.... but don't we want to back them up somewhere?

Looks like another bug. "ubuntu-gtk3-armhf" is the target for Raspbian Bullseye, and it uses gtk3.
On the other hand, "ubuntu-armhf" is not used anywhere, AFAICT, so should be removed.
I'll fix it.

The alpha version has "darwin-wx32", the beta version has "darwin-wx321" and prod version has "darwin-wx32".

[rgleason]

Jon, can this be closed now? I used a link to this in OpenCPN Discussions OpenCPN/OpenCPN#3183