Please provide manual installation steps

[jin-eld]

Hi,

I'd like to package fluidd for Yocto (I am running klipper on various boards using images built with Yocto) and in order to do that I'd either need manual installation instructions or a Makefile with an install target, that would put everything that is needed where it belongs.

I am not familiar with web UIs and their packaging, so I'd appreciate any hints or instructions. The page https://docs.fluidd.xyz/installation/manual says "Manual configuration steps are coming soon.", is there any ETA on that? :)

Cheers,
Jin

[wlhlm]

I'm maintaining an unofficial Fluidd package in the AUR. You can look in there for build instructions:

https://aur.archlinux.org/cgit/aur.git/tree/PKGBUILD?h=fluidd-git

The core of it is to run npm install to fetch the dependencies and then run vue-cli-service build (you can install vue-cli-service externally, or run it from the Fluidd dependency installation via ./node_modules/.bin/vue-cli-service). After that step completes, the final Fluidd files are located in the dist directory and you can just copy them to the desired destination in your Yocto install.

Keep in mind that Fluidd doesn't build with the latest node version, so you need an LTS release (official builds use node 12, but 14 works as well).

[jin-eld]

Thanks a lot, this info is indeed very useful, I think I should manage to reproduce it in a Yocto recipe, I know there's even an npm .bbclass for handling npm related stuff, so keeping my fingers crossed :)

[matthewlloyd]

I'd also like to request manual instructions. I was attempting to do some development work on Fluidd - I installed Node v14, ran npm install, and tried vue-cli-service, but gave up trying to solve this out of memory error after lots of Googling (tried GATSBY_CPU_COUNT=1, --max_old_space_size=8192, etc.):

⠙  Building for production...Starting type checking service...
Using 1 worker with 8192MB memory limit
⠦  Building for production...Warning (worker-plugin): output.globalObject is set to "window". It must be set to "self" to support HMR in Workers.
 ERROR  Error: spawn ENOMEM
Error: spawn ENOMEM
    at ChildProcess.spawn (internal/child_process.js:403:11)
    at spawn (child_process.js:577:9)
    at spawnWithSignal (child_process.js:714:17)
    at fork (child_process.js:129:10)
    at ChildProcessWorker.initialize (.../fluidd/node_modules/jest-worker/build/workers/ChildProcessWorker.js:139:45)
    at new ChildProcessWorker (.../fluidd/node_modules/jest-worker/build/workers/ChildProcessWorker.js:130:10)
    at WorkerPool.createWorker (.../fluidd/node_modules/jest-worker/build/WorkerPool.js:44:12)
    at new BaseWorkerPool (.../fluidd/node_modules/jest-worker/build/base/BaseWorkerPool.js:134:27)
    at new WorkerPool (.../fluidd/node_modules/jest-worker/build/WorkerPool.js:30:1)
    at new JestWorker (.../fluidd/node_modules/jest-worker/build/index.js:112:26)
    at TaskRunner.run (.../fluidd/node_modules/@vue/cli-service/node_modules/terser-webpack-plugin/dist/TaskRunner.js:60:21)
    at TerserPlugin.optimizeFn (.../fluidd/node_modules/@vue/cli-service/node_modules/terser-webpack-plugin/dist/index.js:396:24)
    at AsyncSeriesHook.eval [as callAsync] (eval at create (.../fluidd/node_modules/tapable/lib/HookCodeFactory.js:33:10), <anonymous>:8:17)
    at AsyncSeriesHook.lazyCompileHook (.../fluidd/node_modules/tapable/lib/Hook.js:154:20)
    at .../fluidd/node_modules/webpack/lib/Compilation.js:1409:36
    at AsyncSeriesHook.eval [as callAsync] (eval at create (.../fluidd/node_modules/tapable/lib/HookCodeFactory.js:33:10), <anonymous>:6:1)

[jin-eld]

@matthewlloyd everything worked for me with node12 in Yocto
@wlhlm thanks again for the instructions!

[matthewlloyd]

@matthewlloyd everything worked for me with node12 in Yocto

I tried Node 12 too with the same result :/

[ihrapsa]

@jin-eld I really like your idea of a dedicated yocto image for klipper+moonraker+fluidd. What's your progress on that? Also, what boards are you using? I managed to run everything on OpenWrt (on a 16mb flash, 128mb RAM, 580Mhz mt7688 MIPS device) and although it runs fine it's still bloated with router dedicated packages/modules that take up some memory. I don't have experience with yocto but I'd like to help.
This is the project I was talking about: https://github.com/ihrapsa/KlipperWrt

[jin-eld]

@jin-eld I really like your idea of a dedicated yocto image for klipper+moonraker+fluidd. What's your progress on that?

@ihrapsa
I started with klipper + octoprint a year ago, but quickly disliked octoprint and used my printer via the local LCD, without a web interface. This year someone pitched fluidd, so I thought I'll give it a go; I did package fluidd using the above instructions, but did not have the time to look at moonraker which - of course - does not provide any setuptools configuration which would have made packaging easy.

So right now the image is still without an interface, there's only klipper there.

Another limitation is that I am not installing gcc to cross compile the MCU code, so you'd have to compile the Klipper MCU code on the PC and flash your printer board yourself. Otherwise the image is "ready to run" with all services set up, one only needs to provide the printer.cfg for their printer.

Overall I did not see any interest/feedback regarding that project, so I did only the things that I needed for my personal setup, I am running my printer like that for over a year now.

Also, what boards are you using? I managed to run everything on OpenWrt (on a 16mb flash, 128mb RAM, 580Mhz mt7688 MIPS device) and although it runs fine it's still bloated with router dedicated packages/modules that take up some memory. I don't have experience with yocto but I'd like to help.

The nice thing about Yocto is that the board becomes a don't care as long as Yocto has support for it. I started with an ASUS Tinker Board S, however it had unstable USB, if I understand correctly this is due to a bad hardware/power supply design. I did not dig into details, but simply added a configuration for an older Cubietruck board which I happened to have, also clamped some ferite filters onto the USB cable and did not have any USB issues ever since.

How did you manage to get everything down to 16MB of flash? I think Python itself is quite bloaty in terms of disk space, but I have to admit that I did not optimize for space at all since the hardware I had offered enough resources. Also keep in mind, that some .gcode files can become really big (I do not "feed" .gcode, but find it much easier to use Klippers virtual SD-Card functionality which also works nicely with my local LCD on the Ender 5).

My current cubietruck ext4 image is 86MB

I don't have experience with yocto but I'd like to help.

If you could provide a setuptools patch for Moonraker that'd speed up things a lot, since I did not manage to convince myself to look into it yet :) If I had that, I could integrate it into the meta-klipper Yocto layer easily.
I mean something like this (I had to do this for klippy):
https://github.com/jin-eld/klipper-linux/blob/master/yocto/meta-klipper/recipes-printing/klipper/files/0001-Allow-to-package-klippy-using-setuptools.patch

This is the project I was talking about: https://github.com/ihrapsa/KlipperWrt

This is not really a build, right? I do not see any OpenWRT Makefiles for klipper, moonraker or fluidd?

As for your CPU, I am not familiar with it, I see there is a machine.conf for it, but I do not know of "linkit" is some special variant and if its the kernel that you need. The architecture is supported in Yocto though, so putting the bits and pieces together in a new machine.conf should be possible.

https://github.com/micro-iot/micro-iot/blob/master/meta-linkit7688/conf/machine/linkit7688.conf

If you'd like to try things out, I could look into importing the meta-linkit7688 layer and building an image for it, but I'd probably leave the size optimizations to you :)

[ihrapsa]

@jin-eld Wow thanks for the detailed response.

How did you manage to get everything down to 16MB of flash?

I should've mentioned that I used extroot for everything to fit in. I only built the OpenWrt .bin file with the necessary kernel modules and that was just ~5mb. The rest gets installed on a microsd card that is setup as external root. The whole process is manual and was adapted from the Mainsail Manual Setup guide. For fluidd the process was identical, the only differences were the static web server files and the nginx configurations.

I started with klipper + octoprint a year ago, but quickly disliked octoprint and used my printer via the local LCD, without a web interface

I started with octoprint as well since there were some already successful proofs online that's doable but the performance was terrible and that lead me to klipper and its lightweight webservers.

I started with an ASUS Tinker Board S, however it had unstable USB, if I understand correctly this is due to a bad hardware/power supply design.

This device I have seemed to have similar issues with occasional disconnections due to EMI. I'm now very careful where and how I run my cables and the issue kind of disappeared. I might have to upgrade to proper shielded cables +/- ferrite cores as well.

If you could provide a setuptools patch for Moonraker that'd speed up things a lot, since I did not manage to convince myself to look into it yet :)

Unfortunately, I don't have any experience with python packaging but I found this Moonraker AUR PKGBUILD script if it's of any help. :)

This is not really a build, right? I do not see any OpenWRT Makefiles for klipper, moonraker or fluidd?

No. The repo focused on the manual installation at first but now I have also released the extroot filesystem with everything installed that makes the whole process faster and simpler (just untarring the archive on an sdcard and setting up extroot on the box).

Regarding OpenWrt makefiles, I found someone (/andryblack) who already made one for klipper a while ago but none for Moonraker or fluidd. I was considering to go this way sometime but I've encountered some memory issues (~80% usage and sometimes more) that made me look for ways to optimise RAM usage (some options were ram/zswap, stripping down router related or unneeded packages, and Yocto). I found Yocto to be the best in terms of performance but also a lot more complicated... and that's how I found about your work.

If you'd like to try things out, I could look into importing the meta-linkit7688 layer and building an image for it, but I'd probably leave the size optimizations to you :)

Wow thanks! One thing though, (I've never tinkered with yocto yet but really plan to so I've got some questions) Is it possible to build a yocto image that would automatically enable an sd card as extroot? Or is this something that needs to be addressed in u-boot to boot the kernel from the sdcard instead of the NOR flash?

Also, This MIPS box has a weird Boyamicro BY25Q128AS flash chip that apparently keeps the OpenWrt devs from merging this PR(due to some issues with manufacturer and device ID). Will this cause any issues for Yocto support?

Thanks again!!

[jin-eld]

@ihrapsa

How did you manage to get everything down to 16MB of flash?

I should've mentioned that I used extroot for everything to fit in. I only built the OpenWrt .bin file with the necessary kernel modules and that was just ~5mb. The rest gets installed on a microsd card that is setup as external root.

OK, that makes more sense, Python with all its modules is quite "heavy".

Unfortunately, I don't have any experience with python packaging but I found this Moonraker AUR PKGBUILD script if it's of any help. :)

I did not have any experience with python packages either when I had to package Klippy, so I had to learn it, because nobody would do it for me ;)

The AUR approach would work too though, seems its enough just to copy the files to the appropriate location and tune the service script, should be easy enough.

Regarding OpenWrt makefiles, I found someone (/andryblack) who already made one for klipper a while ago but none for Moonraker or fluidd. I was considering to go this way sometime but I've encountered some memory issues (~80% usage and sometimes more) that made me look for ways to optimise RAM usage (some options were ram/zswap, stripping down router related or unneeded packages, and Yocto). I found Yocto to be the best in terms of performance but also a lot more complicated... and that's how I found about your work.

Yocto is a build system for embedded distributions, OpenWRT is a build system for a more dedicated distro, but essentially you can tell both what you want to package into the rootfs and how things should be set up. I'd say Yocto is more flexible and gives you better control of your build, but in itself it will not increase performance of the image - it's up to you what you package and how you set stuff up.

Wow thanks! One thing though, (I've never tinkered with yocto yet but really plan to so I've got some questions) Is it possible to build a yocto image that would automatically enable an sd card as extroot? Or is this something that needs to be addressed in u-boot to boot the kernel from the sdcard instead of the NOR flash?

I am not familiar with your device, depending on device support Yocto also spits out u-boots and kernels where things can be configured as needed. Don't celebrate yet, we do not know how good the device is supported in that layer and if it matches your particular one at all. Debugging and making sure it actually works would be up to you, since you're the one who has the hardware.

Also, This MIPS box has a weird Boyamicro BY25Q128AS flash chip that apparently keeps the OpenWrt devs from merging this PR(due to some issues with manufacturer and device ID). Will this cause any issues for Yocto support?

This is not a Yocto, but a Linux support issue, I see that in the above PR they are tuning the device tree to enable support for some hardware components. I am not a low level guy, so that's beyond my knowledge level. You can apply any patches you like on Yocto, just like you can do so with OpenWRT, if these patches work or not is a different story - the question there is "Is this hardware supported by Linux?"

I guess we should take this discussion to https://github.com/jin-eld/klipper-linux since we now ended up spamming fluidd with off topic issues?

[pedrolamas]

This is not something that we want to maintain ourselves, so I would recommend just using Kiauh for manual installs.

You can see the steps it takes to install Fluidd here: https://github.com/th33xitus/kiauh/blob/master/scripts/install_klipper_webui.sh

[jin-eld]

@pedrolamas

You can see the steps it takes to install Fluidd here: https://github.com/th33xitus/kiauh/blob/master/scripts/install_klipper_webui.sh

"packaging" a project and some custom on-system install script are two totally different things. The latter only works on a live system, while the first allows you to package a project for any target system, including cross-compile environments. Also, if a projects wants to be part of a default distro, be it Debian, Fedora or alike, so that users could simple dnf install/apt-get install the package on their Raspbian systems or anywhere else really - then a generic "make install" that at the very minimum honors DESTDIR is really a must.

Failure to provide project build/install scripts is the reason why projects like Kiauh exist in the first place. Projects like klipper who also completely ignore the packaging topic can be happy that someone was annoyed enough to spend time on custom installers. In reality, it's work that should have been done in the project itself: imagine you'd pull some C open source project and only get source files, but no compile scripts/no Makefiles... which user would want to mess with that?

Anyway, if you don't want to do this, it's your call, imho its a meaningful feature request which will simplify packaging and installation of your project on any Linux system and thus help it to spread, but it's up to you to accept or reject it.

[pedrolamas]

@jin-eld I have nothing against packaging apps, but the reality is that at the moment we have limited resources and we need to work out the priorities in order to keep moving.

The points you make are very good and I think I can agree with all of them, but I personally don't have the time or knowledge necessary to do it myself.

Having said that, this is an open-source project and we would definitely welcome any PRs that would help get an installable package out there!

In the end, this is just another web app, so it should follow the basic "install nodejs, npm i, npm run build, serve" workflow!