Elgg, Lokole [also on 21.04] & Node-RED [nodered] don't yet work on Ubuntu 21.10 [Impish Indri] ?

[holta]

Not at all urgent but something to monitor and be aware of as the OS's 2021-10-14 release date approaches.

The reasons are likely entirely disconnected from PHP 8.0 — these 3 app failures are just early data points — that arose during the very extensive IIAB testing as part of PR #2832, providing an early glimpse of Ubuntu Server 21.10.

Tangentially related:

• Testing IIAB with PHP 8.0, part of new OS's like Ubuntu 21.10 pre-releases #2814
• Test IIAB on Ubuntu 21.10 (RPi+PC) x (Server+Desktop). Does RPi WiFi firmware 7.45.18.0 work well? [OS released 2021-10-14] [JupyterHub install pip fails intermittently?] #2818

cc: @nzola

[holta]

A bit more detail:

1. http://box/elgg - browser shows the following when IIAB's installed on Ubuntu 21.10: (whereas it works on Ubuntu 21.04)

   HTTP ERROR 500

2. http://box/lokole - browser shows the following: (regardless whether IIAB's installed on Ubuntu 21.10 / PHP 8.0 OR on Ubuntu 21.04 / PHP 7.4)

   502 Bad Gateway
   nginx/1.18.0 (Ubuntu)

3. http://box/nodered - shows the following when IIAB's installed on Ubuntu 20.10: (whereas it works on Ubuntu 21.04)

   Service Unavailable
   The server is temporarily unable to service your request due to maintenance downtime or capacity problems. Please try again later.
   Apache/2.4.46 (Ubuntu) Server at 127.0.0.1 Port 8090

[jvonau]

nginx logs? apache logs?

[holta]

@jvonau apologies I typo'd the subject line (this is about Ubuntu 21.10 not 20.10).

The original VM is now lost but a new one is now building here:

198-u2110-srv-BIG-0629 = 10.8.0.26

And should hopefully complete well within 45min. Feel free to login and reboot it as nec.

[holta]

198-u2110-srv-BIG-0629 = 10.8.0.26

Its iiab-diagnostics are here: http://sprunge.us/q2aHVz?en

Don't hesitate to log in if you choose to.

PS all 3 page failures are re-confirmed on this new VM, per the subject line / title of this ticket (i.e. for Elgg, Lokole and Node-RED).

[jvonau]

#2986 #2739

[holta]

Moved to the IIAB 7.2 milestone, to help @nzola in his decision-making in coming days, regardless what is decided.

(e.g. even if Lokole cannot be made to work on Ubuntu 21.04 in a short timeframe, as @rickgurmet is hoping for!)

[jvonau]

Given lokole works on the primary OS that nzola has deployed, proven from #2988 with 46-rpi4-lite & 42-u20 across the vpn but what is missing is a current report from a u-21.04 install for a quick no go/go test and easy access to probe further.

[holta]

Given lokole works on the primary OS that nzola has deployed, proven from #2988 with 46-rpi4-lite & 42-u20 across the vpn but what is missing is a current report from a u-21.04 install for a quick no go/go test and easy access to probe further.

@rickgurmet can we give @jvonau remote access to your Ubuntu 21.04 machine where Lokole was attempted?

Email or Text me directly, if possible, to arrange password etc?

(Or if that proves too complicated for you, I can set up an Ubuntu 21.04 VM to do similar.)

[holta]

FYI @jvonau confirmed on 2 different Ubuntu Server 21.04 machines that reinstalling Lokole seems to work, as follows:

sudo systemctl stop supervisor.service
sudo rm -rf /home/lokole
cd /opt/iiab/iiab
sudo ./runrole --reinstall lokole

Unclear why the original Lokole install does not work — upon being installed by IIAB (http://box/lokole shows "502 Bad Gateway") — until the above 4 commands are run.

But the good news is that the above workaround would appear to be repeatable (reproducible) for those who need it.

[nzola]

FYI @jvonau confirmed on 2 different Ubuntu Server 21.04 machines that reinstalling Lokole seems to work, as follows:

sudo systemctl stop supervisor.service
sudo rm -rf /home/lokole
cd /opt/iiab/iiab
sudo ./runrole --reinstall lokole

Unclear why the original Lokole install does not work — upon being installed by IIAB (http://box/lokole shows "502 Bad Gateway") — until the above 4 commands are run.

But the good news is that the above workaround would appear to be repeatable (reproducible) for those who need it.

Excellent!

[deldesir]

Lokole workaround works on Ubuntu server 21.10 on RPI4. Nodered still asks for its own workaround.

[holta]

Lokole workaround works on Ubuntu server 21.10 on RPI4. Nodered still asks for its own workaround.

Thanks @deldesir

If possible please post the sprunge.us URL that is output when you run sudo iiab-diagnostics

(This greatly helps everyone contribute towards understanding & fixing things!)

[holta]

Node-RED 2.1.0 was release overnight:

@deldesir let's check in coming days...to ascertain if this helps on Ubuntu 21.10 etc?!

[holta]

Node-RED 2.1.1 was released today... try that instead of yesterday's 2.1.0 ?

[holta]

Node-RED 2.1.1 was released today... try that instead of yesterday's 2.1.0 ?

As is auto-installed by BIG-sized IIAB installs, or cd /opt/iiab/iiab ; sudo ./runrole nodered if these 2 vars are set in /etc/iiab/local_vars.yml :

nodred_install: True
nodred_enabled: True

NOTE: Node-RED's error last week ("502 Bad Gateway") is now different from its original June 29th error ("Service Unavailable") near the very top of this ticket: #2833 (comment)

[holta]

Testing with Node-RED 2.1.2 is now appropriate:
https://github.com/node-red/node-red/releases/tag/2.1.2

[holta]

@deldesir please see @jvonau's possible Node-RED fix here:

• Node-RED on NGINX not Apache #2843 (comment)

[holta]

1. @jvonau has helped trace the evolution of /etc/nginx/conf.d/nodered-nginx.conf in January of 2020 (20 months ago) here:

   • Refine @jvonau's "Move nginx shims to roles" #2131 #2137 (comment)
   • Stub /etc/nginx/conf.d/nodered-nginx.conf to port Node-RED natively to NGINX? (http://box/nodered) #2154 (comment)

2. FWIW the above does not seem to be the problem. As no changes were needed to nodered-nginx.conf as I was able to get http://10.8.0.50/nodered to work (on Ubuntu Server 21.10 on Raspberry Pi 4) using these two steps I just now tried:

   1. I installed Node.js 16.12.0 (which includes /usr/bin/npm 8.1.0, so no need for the npm apt package!) I don't know why this was not installed. @deldesir might know as he was working on this earlier, but in any case it's now fixed, so the point is probably moot.
   2. I ran ./runrole nodered to install Node-RED 2.1.2 which worked great.

   CLARIF: I ran apt purge npm to be sure, to clear out all residue from the earlier npm 7.5.2 that apt had installed, but that purging step was likely not necessary. Thanks to everyone (@jvonau and @deldesir) who pushed this forward in recent days. It seems likely (?) that Node-RED issues are all now solved. But before we declare victory, I'd ask @deldesir to please test carefully on a couple more fresh installs...so we can all be sure.

[holta]

Possibly we should should close this ticket very soon?

Especially if it's confirmed that:

1. Node-RED is working well on all OS's, and
2. Lokole's "install it twice" workaround might possibly soon be understood (and/or automated?)

[holta]

FYI Node-RED 2.1.3 is now released:
https://github.com/node-red/node-red/releases/tag/2.1.3

(Presumably these near-daily releases will calm down soon, as most/all of these little 2.1.x bugs have now been patched...)

[holta]

@deldesir wrote:

Lokole workaround works on Ubuntu server 21.10 on RPI4.

I confirm that Lokole's manual reinstall workaround also works on these 2 Ubuntu 21.10 VM's (one server, one desktop) here:

• 196-rpi4-U2110-serv-BIG = http://10.8.0.74/lokole = http://sprunge.us/xmoIOs?en
• 197-rpi4-U2110-desk-BIG = http://10.8.0.30/lokole = http://sprunge.us/e8Q4Av?en

cc: @nzola

[holta]

@deldesir wrote:

Lokole workaround works on Ubuntu server 21.10 on RPI4.

I confirm that Lokole's manual reinstall workaround also works on these 2 Ubuntu 21.10 VM's (one server, one desktop) here:

• 196-rpi4-U2110-serv-BIG = http://10.8.0.74/lokole = http://sprunge.us/xmoIOs?en
• 197-rpi4-U2110-desk-BIG = http://10.8.0.30/lokole = http://sprunge.us/e8Q4Av?en

cc: @nzola

I've confirmed that Lokole's 4-line manual reinstall workaround also works on Ubuntu Desktop 22.04 (pre-release daily build on x86_64 VM).

[jvonau]

#2843 (comment)

[holta]

Just a clarification of what was mentioned ~2.5 days ago — http://10.8.0.22/lokole on Ubuntu Desktop 22.04 (pre-release daily build on x86_64 VM) appears to fully work after @jvonau's 2021-09-30 workaround is run: (which boils down to installing Lokole twice)

• Elgg, Lokole [also on 21.04] & Node-RED [nodered] don't yet work on Ubuntu 21.10 [Impish Indri] ? #2833 (comment)

[holta]

Just FYI I've posted a link to the Lokole workaround here, so more people who need it can find it:

https://github.com/iiab/iiab/wiki/IIAB-7.2-Release-Notes#known-issues

Lokole does not work when installed on Ubuntu 21.04 or 21.10, until you apply the #2833 workaround.

[jvonau]

Unsupported OS's close

[holta]

Unsupported OS's close

Agreed: while Ubuntu 21.10 is tolerated for a little while longer, Ubuntu 22.04 pre-releases are now center stage.

No question you and @nzola have put a lot of work into this, and we can only do our best!