Volunteer Plugin Testing

[rgleason]

###The Problem:

The problem is that we will release a new version of the Testplugin Template or even ShipDriver Template, then try building all the OS metadata and tarballs for one of the plugins and simply hope that all the OS work, because we do not have the means or equipment to test them all.

Then it gets worse, we think they are working, and build all the rest of the plugins using the new template. Then the reports dribble in, first Flatpak arm64 is failing. So we fix that and rebuild all the plugins. Then we find out another OS build is failing. What are we going to do, rebuild all the plugins again? No. We are going to remove all the plugins for that OS.

The Start of a Solution

Develop a reliable testing program. Taking it one step at a time.

When Jon releases a new Testplugin Template he pushes the commit to testplugin_pi and increments the version. https://github.com/jongough/testplug...commits/master

When he pushes the commit that starts a Circleci build process which should create all the necessary metadata and tarballs for all the currently supported OS which are deployed to Cloudsmith and stored there in

https://cloudsmith.io/~opencpn/repos...prod/packages/ and sometimes in the testplugin-beta repository. He may even add the results to the Plugin Manager "beta" catalog.

When this is done, the first thing we need is for all testers to download their OS version of the testplugin tarball and use the Options > Plugins > "Import Plugin" feature (or use the beta catalog) to test the installation, enabling, preferences, menus and general operation of the plugin making sure OpenCPN does not crash, in a reasonably timely manner.

Once we are reasonably certain that the new template is working for all OS, I will, over a period of weeks, update all of Sean's plugins, plus some others, pushing them to github and deploying to Cloudsmith. Each plugin will then be added to the Plugin Manager "Beta" catalog for testing by all the OS Testers. We will want you all to test the results on your OS and report back in a timely fashion.

Then the plugins will be added to the PIM master catalog.

Thus we will avoid needless extra and repetitive work. If there are any ideas or suggestions to improve this process please write below. Thanks.

I do hope other volunteers will commit to this testing process.

Needs

1. Alerts: We need a good way to alert all volunteer testers that there is a new plugin test needed and what plugin which will be located on the beta catalog.

2. Sign off: We need a good method of registering and keeping track of volunteer testers results, all in one central location that everyone can quickly view. Ideally I would like a form listing the plugin all OS that testers could use to sign off on a plugin.

3. Problems: If there is a problem they can use the plugin's Issues to report.

[rgleason]

There is an Issue under OpenCPN/plugins related to this topic which will be closed at some point. It is linked to here for reference Plugin Testing

There is also a Cruiser Forum Opencpn topic Volunteer Plugin Testers Needed!

[ALROCHARARI]

Hello,

This is my first post and I expect it to be at the right place.
I also left the description of failure in the sheet template.

WeatherFax from Beta catalog test on Win11:

• (OpenCPN 5.6.2)
  
    1- When trying to download WeatherFax 1.9.30.11 from Beta catalog: 
    "Download error: The requested URL retuned error:404 Not Found
    Please check system log for more info." 
  

WeatherFax from Beta catalog test on Debian Bullseye/11 rpi:

• (OpenCPN 5.6.2-0+b75be76)
  
    1- Settings window always starting "minimised" on top left of the screen and requesting each time stretching to be used 
    2- No plugin available in Beta catalog for WeatherFax
  

• (OpenCPN 5.6.2)
  
    1- Plugins Beta catalog empty except WMM 1.1, ChartDownloader 1.4, Dashboard 1.2 and GRIB 4.1
  

Can't wait to test new version of WeatherRouting which is failing (also crashing OpenCPN itself) each time I try to use it (as well on Win11 as on Debian Bulseye11 rpi)

Hope it helps

[rgleason]

Did you update both the PIM Beta Catalog and the PIM Master Catalog?
Those are outdated versions no longer available in the current PIM catalog.
You should be looking for v1.9.32.0 in the Beta Catalog.
Sorry for any confusion.

Master Catalog has v1.9.31.0 I believe.

[ALROCHARARI]

What is PIM?

Yes, I did the updates of Master & Beta catalogs multiple times.
On Windows11, I confirm only WeatherFax 1.9.30.11 is visible as well in Beta as in Master.
On Bullseye11 I confirm that I don't see WeatherFax plugin in my FP installation in the Beta catalog as in the Master one. On the other installation, I only have 4 standard plugins visible and installed but no WeatherFax.

Give me the time to come up to speed. I'm still afraid to do something wrong and giving you wrong feedback.

[rgleason]

Very sorry. the Pull Request to opencpn/plugins Beta catalog has not been merged yet, You will see it when it is merged.
You are doing fine!
PIM is my shorthand for Plugin Manager. I am not going to type it all the time.

EG: Under Toolbar Options > Plugins (PI) > Pick "Settings" then select Beta. Pick Done.
Then pick "Update Plugin Catalog: Beta" then you must install and then enable new plugin versions found.

[ALROCHARARI]

Tested again on 3 environments:
Windows11 msvc10.0.14393: Found version 1.9.32.0. Installation failed: file not found 404
Bullseye11 flatpak-aarch64:20.08: Found version 1.9.32.0. Installation failed: file not found 404
Bullseye11 ubuntu-gtk3-arm64:20.04: Packages not available

On Bullseye11 flatpak-aarch64:20.08, can I maybe reset manually parameters to avoid the Options/Settings window minimised on top left corner of the screen systematically (On first opening after OpenCPN startup)? Issue not present in ubuntu-gtk3-arm64:20.04.

I'm using a few o-charts maps, how to avoid loosing the licenses when I fully reinstall the PI ?

[rgleason]

Thank you. This has been a moving target. I am sorry. The old version got rewritten by newer ones, due to background stuff happening. My apologies. Will advised.

[rgleason]

Ok, we have a new vesion Weatherfax 1.9.32.1 in the Beta Catalog.
In Options > PI > Settings, select "Beta" Then "Update Plugin Catalog: Beta"
Then go to weatherfax in the list and Upgrade to 1.9.32.1 and then "Enable" the plugin.

If there is trouble with Image Wizard popping up instead of just automatically finding the proper coordinates and overlaying on charts., then you need to Opencpn and to remove the old coordinateSets.xml from your user directory (not the installation directory, which has the new coordinateSets.xml.) Then startup of OPenCPN and the Plugin should cause the new coordinateSets.xml file to be copied to your user directory where it is normally read. If you are having trouble with this, let me know. We have some fixes in mind to mak this better.

Thank you for testing! It is a vital part of having good working plugins.

[ALROCHARARI]

Finally happy to report positive test results.

I suggest to continue to enrich the shared google document and add as much details as possible into it.

I'm looking to install other platforms just to provide better support in this test phase. (Updating my Ubuntu 18.02). Will keep you updated.
What are for you the key platforms you want to test positively before moving forward with other evolutions or deployments?

[rgleason]

Alrocharari Thanks for your thorough spreadsheet report. I see that "armhf debian bullseye/11 gtk3 rpi" plugins Beta Catalog is empty except for WMM.

As Dave notes:

"All" should be the target, but reasonably one should prioritize on the platforms that are known to be in high demand. (Windows, MacOS, ubuntu focal/20.04, rpi armhf, rpi arm64, Android, debian buster/10, debian bullseye/11.)"

For Weatherfax we need "Basic Testing" done as described briefly in the OpenCPN Plugin Testing Spreadsheet "Instructions" Tab
https://docs.google.com/spreadsheets/d/19JvjTHTXqzE-atlgQ_SPdqTaAxfCUAiQ5_1vXa1ENb0/edit#gid=2104338124

Also the Internet Retrieval URL's have all been updated by Ozolli and should it should be confirmed that the weatherfax overlays work as expected. Report any specific problems. I have not found any yet.

Confirm that you can Retrieve > Update Data. Do we need a confirmatory message on completion?

Do we need a prompt "Copy installation CoordinateSets.xml over the user version? Yes No"

[ALROCHARARI]

I confirm that gtk3 plugins Beta Catalog is empty except 4 mentionned default plugins.
Retrieve & update work fine on all platforms tested. I would suggest to enrich the google spreadsheet with specific sheets per plugin with the test stages to complete in different columns plus a free comments area.
/ Copy installation CoordinateSets.xml over the user version? Yes No" / should be in the detailed corresponding sheet.

I have now an installation of Ubuntu 22.04 Jammy Jelly Fish. This is my test report:
1- WeatherFax 1.9.32.1 - When trying to recover capture: "failed to configure capture". Window still opening but without title and missing part of the information except about decoding and image correction. After playing with internet fax loads, coming on this option makes OpenCPN crashing immediately.
2- WeatherFax 1.9.32.1 - When refreshing data, crashing after a certain number of updates. Crashing OpenCPN completely. The crash happens only after some internet loads. Not when empty. Not happening all the time! Did load of ECMWF / UK / 96h 10m Wind. Strange beheaviors when coming back on the screen of internet loads with servers names dissapearing when clicking on them. It can make sense but is confusing as it means a selection is already ongoing but is not visible
3-Tactics 1.0.31.1 - Not available, only 1.0.30.0 available from the Beta catalog. 1.0.30.0 working fine except I'm not sailing actually and can not check accuracy of data updates.

BTW: great work by you guys.
Good luck for the next steps.

Raoul

[ALROCHARARI]

Please have a look at additional tests results on new platform (Debian11 on PC and Ubuntu gtk3 on PC).
See also updates on spreadsheet. Please confirm this is in line with your expectations (See new sheets for test protocols)

[rgleason]

Thank you Alrocharari , we are noting your reports and have reffered them to the team.

[rgleason]

Would all test volunteers please try

weather_routing 1.13.42 after being merged into the Beta Catalog.
weatherfax 1.9.33 which has been merged into the Beta Catalog.

We are particularly interested in testing of the flatpak builds!
Thank you.

For weatherfax_pi we suggest you uninstall weatherfax first, close opencpn. Then determine that your opencpn user writable directory does not have an opencpn/plugins/weatherfax directory. Then go ahead and install the new version.
When downloading a fax, if the overlay does not occur and you are forced to use Image Manager, please advise. It is likely you have not removed the old user version of coordinateSets.xml. Once you do this it should work. Later we are going to try to remedy this situation in a user friendly way.

Please report the results of your tests in the Testing Spreadsheet or in the Cruiser's Forum Weatherfax thread or the Weather_routing Thread

Thank you! We depend on your reports to confirm operation of these plugins!

[rgleason]

We need the beta version of watchdog tested.

https://www.cruisersforum.com/forums/f134/watchdog-plugin-150313.html#post3674963

Watchdog 2.4.45.2 master-chg2sean -fix divergence in branch and add Pypilot.
Volunteer Testers,
We need to have watchdog_pi v2.4.45.2 Tested, particularly for the the new pypilot functions from Sean's repository.
Watchdog_pi v2.4.45.2 will be found in the PIM Beta directory once bdbcat merges my pull request and you update your Beta Catalog. Please post any issues in the Watchdog Thread or in the Test spreadsheet.

[bdbcat]

There is no PR pending for OCPN plugins.

[rgleason]

Yes, that is correct. You accepted it. Very Sorry about that.