Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Plugin manager: some erratic installs #847

Closed
macmpi opened this issue Nov 15, 2016 · 20 comments

Comments

Projects
None yet
5 participants
@macmpi
Copy link
Contributor

commented Nov 15, 2016

Hi,

I encountered erratic plugin installs several times, particularly when installing a dropped zip in install UI.
Indeed it seems that sometimes installation may happen twice (the second would fail if the first one was successfully registered).
It may be due to insufficient clean-up of temporary install files upon installation failure.
If some install procedure fails, plugin manager may discover remainings of /tmp/downloaded_plugin.zip and try to install over-again (an exemple here).

Maybe it would be suitable to remove each temporary file & folders after each step even if not successful (/tmp/downloaded_plugin.zip, /tmp/downloadedPlugin folder, and /tmp/plugins folder) or find a way to avoid those multiple installs.

@volumio

This comment has been minimized.

Copy link
Owner

commented Nov 15, 2016

Yes, I would say we should understand why installation restarts. Any idea?

@macmpi

This comment has been minimized.

Copy link
Contributor Author

commented Nov 15, 2016

I'm afraid no idea at this time.
However /tmp cleanup would at least avoid consequences, while freeing-up some RAM in the meantime.
Re-entry could still be noticed with journalctl messages, so problem does not go unnoticed.

@macmpi

This comment has been minimized.

Copy link
Contributor Author

commented Nov 15, 2016

Isn't the issue with this, which may be called several times in pages refreshes?

After first transfer to /tmp/plugins/ and install upon socket.emit('installPlugin', { url:pluginurl});, installPlugin does not delete original file from /tmp/plugins/after install.
Therefore any re-entry in app.route('/plugin-upload') will produce re-install.

@macmpi

This comment has been minimized.

Copy link
Contributor Author

commented Nov 16, 2016

While at it, it would be nice if RAM usage can be optimized by limiting plugin duplication in /tmp: indeed installing some big plugins (like spotifyconnect for arm6) on limited RAM devices (such as rpi0) may lead to fairly low RAM conditions.
Here are some suggestions, but you may have better ideas.

For drag&drop install, can't the initial upload be directly into /tmp/downloaded_plugin.zip (omitting temporary /tmp/plugin/myplugin.zip), and then just test inside installPlugin() from pluginmanager.js, if that file already exists (or empty URL passed) to differentiate drag&drop or network install (and do wget only in the latter case)?
Then, can /tmp/downloaded_plugin.zip extraction happen directly into /data/plugins/tmp_plugin (and delete /tmp/downloaded_plugin.zip right-away), and finally move tmp_plugin into final destination with final name once package.json is actually read (and no error found)?

Those (or similar) measures could significantly save RAM in some cases, and by cleaning after each step, would also be safer, while the re-entry issue is being investigated in drag&drop situations.
Thanks for consideration.

@volumio

This comment has been minimized.

Copy link
Owner

commented Nov 17, 2016

This is really a good idea. Give e some time to implement it

@simonspa

This comment has been minimized.

Copy link

commented Dec 11, 2016

I'm having the same issue with this. Especially when the installation of the plugin takes a while (calling sudo apt-get update in a RPi takes quite long), a second instance is spawned during the installation process...

@balbuze

This comment has been minimized.

Copy link
Contributor

commented Dec 14, 2016

Yes, volumio 2.041 problem is still here. If plugin needs to download a file, we can see several download (each time the plugin installation restarts, a new download starts). Even aif at the end the installation works, it takes twice the time, and cpu to unzip the file.

@xipmix

This comment has been minimized.

Copy link
Contributor

commented Mar 22, 2017

The double-install seems to be some kind of failure to clear state in the UI.

My perception, which could well be wrong is as follows. On a linux box with xfce & firefox, I can make it do two uploads by performing one, and then clicking again with my left mouse button on a tab in the browser or even in a terminal window. It seems to be that somehow the UI picks up the event and reruns the upload. I am always doing the upload by clicking in the box and selecting a file from the 'file open' popup, not dragging and dropping. I have managed to make it happen more than once too but I am not really sure how.

@balbuze

This comment has been minimized.

Copy link
Contributor

commented Mar 22, 2017

I'm always doing like you do. Clicking the file. Firefox on xubuntu. And sometimes it starts twice. Even if the plugin install fails, if no action (closing the dialogue), after a while it restarts...

@xipmix

This comment has been minimized.

Copy link
Contributor

commented Mar 23, 2017

@macmpi I think I can see a way to implement direct reuse of the file the user uploaded, which will save some space in /tmp. The patch depends on changes made in #1083 so a PR will have to wait for volumio to have some time to consider & perhaps merge that.

@macmpi

This comment has been minimized.

Copy link
Contributor Author

commented Mar 23, 2017

nice!
Any clue on the "reload twice" thing?
BTW I've not checked if the same issue happens when uploading backgrounds from Appearance panel: might give clues.
(I remember once, I had a plugin "magically" land into the background folder!...)

@xipmix

This comment has been minimized.

Copy link
Contributor

commented Mar 23, 2017

@macmpi

This comment has been minimized.

Copy link
Contributor Author

commented Jun 16, 2017

Looking at it more closely, while working on #1220
It turns-out initial upload happen twice: for each dragged & dropped plugin, 2 files are generated in /tmp/plugins
So following that, plugin install logically happens twice.

So we "just" need to find-out why drag & drop upload happens twice originally...
Does not seem a pluginmanager issue then...

@xipmix

This comment has been minimized.

Copy link
Contributor

commented Jun 17, 2017

In #1105 I noticed the plugin is uploaded by the user, then reuploaded to the express server on port 3000. Are you saying that the d&d method does the first upload twice?

@macmpi

This comment has been minimized.

Copy link
Contributor Author

commented Jun 17, 2017

Sorry my bad, I was misinterpreting something across several tests.

@macmpi

This comment has been minimized.

Copy link
Contributor Author

commented Jun 27, 2017

interestingly with #1231, duplicate installs (per say) seem fixed: this is probably due to more aggressive temporary files cleanup.
However, dialog message still reports install twice: so the underlying "drop zone" issue is still there, while the most annoying consequence is fixed.

Maybe this could help (last note)?

@macmpi

This comment has been minimized.

Copy link
Contributor Author

commented Sep 21, 2017

issue is still here as of v2.278.
Web UI does call pluginmanager twice in a 2sec intervall:

Sep 21 11:32:19 Volumio volumio[860]: Uploading: cdio_paranoia.zip
Sep 21 11:32:19 Volumio volumio[860]: Created safe filename as '4cef67b8-b26f-4e4b-bd50-55cca9efa8a9.zip'
Sep 21 11:32:20 Volumio volumio[860]: Upload Finished of cdio_paranoia.zip as 4cef67b8-b26f-4e4b-bd50-55cca9efa8a9.zip
Sep 21 11:32:20 Volumio volumio[860]: info: Downloading plugin at http://127.0.0.1:3000/plugin-serve/4cef67b8-b26f-4e4b-bd50-55cca9efa8a9.zip
Sep 21 11:32:20 Volumio volumio[860]: info: END DOWNLOAD: http://127.0.0.1:3000/plugin-serve/4cef67b8-b26f-4e4b-bd50-55cca9efa8a9.zip

Sep 21 11:34:20 Volumio volumio[860]: Uploading: cdio_paranoia.zip
Sep 21 11:34:20 Volumio volumio[860]: Created safe filename as '40608650-7660-4db4-94b4-d583810a4afd.zip'
Sep 21 11:34:26 Volumio volumio[860]: info: Fetched 17.9 MB in 1min 24s (212 kB/s)
Sep 21 11:34:26 Volumio volumio[860]: Upload Finished of cdio_paranoia.zip as 40608650-7660-4db4-94b4-d583810a4afd.zip
Sep 21 11:34:26 Volumio volumio[860]: info: Downloading plugin at http://127.0.0.1:3000/plugin-serve/40608650-7660-4db4-94b4-d583810a4afd.zip
Sep 21 11:34:27 Volumio volumio[860]: info: END DOWNLOAD: http://127.0.0.1:3000/plugin-serve/40608650-7660-4db4-94b4-d583810a4afd.zip

Happened same with either drag&drop of click-download from macOS/Chromium/FF (could be platform/browser dependant maybe).

@balbuze

This comment has been minimized.

Copy link
Contributor

commented Sep 22, 2017

Yes I can confirm the behaviour. And other point is that if you install a plugin when an other plugin is installed, at the end of the install, the list shows a new line but the plugin name is the other plugin. A page refresh solve this.

@macmpi

This comment has been minimized.

Copy link
Contributor Author

commented Mar 7, 2018

I guess this is not happening again since local upload/dropbox UI has been removed in favor of volumio plugin install manual command...
Will close, but @balbuze @xipmix let me know if you still see it.

@volumio

This comment has been minimized.

Copy link
Owner

commented Mar 26, 2018

Fixed

@volumio volumio closed this Mar 26, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.