Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Best strategy for preparing multiple IIAB servers using imaging #3247

Open
andrewclark5 opened this issue Jun 9, 2022 · 134 comments
Open

Best strategy for preparing multiple IIAB servers using imaging #3247

andrewclark5 opened this issue Jun 9, 2022 · 134 comments
Labels
Milestone

Comments

@andrewclark5
Copy link

andrewclark5 commented Jun 9, 2022

@tim-moody that should work, but you could do '/library-ext /library bind' and not have the extra level. Think the 'copy function' would just see the unified underlying filesystem and the second target drive would still be mounted in /media/. Once /library is populated on the 'master data drive' mounted via fstab one could just copy the contents to a new target drive, then redeploy on a second machine with just the fstab entries without the need for symlinks. What needs to be on the cloned target is the library.xml from the donor to match what is on the '/library partition' that would be #3052, to save manual intervention so library.xml travels with the data in /library.

Originally posted by @jvonau in #3106 (comment)

@jvonau @tim-moody @shanti-bhardwa

I'd be grateful for final guidance on how best to configure systems for easy deployment bearing the following in mind:-

  1. Maintain separate partitions/drives for Library data and OS/Programs
  2. Be able to restore separate images for each and point IIAB to where to find its /library (and set appropriate edits in fstab)
  3. Ensure that all necessary Library content is contained within its image. For example mention is made by @jvonau in the quote above that a libray.xml file must also be copied. Which one is that? I have found :-
    a. library.xml in /library/zims
    b. iiab-library.xml in /opt/iiab/iiab-admin-console/assets
    ( as well as a deprecated one which I assume I can ignore.)

a. is no problem as it will be copied with /library.
b. would be more problematic...

@jvonau
Copy link
Contributor

jvonau commented Jun 9, 2022

Perhaps you could share your understanding of what is needed and the tools that you are going to use to support your endeavor. My guidance is recorded in #3106, 'b' is not a problem if you make both data and os 'images' at the same time from the source machine.

@tim-moody
Copy link
Contributor

b. iiab-library.xml in /opt/iiab/iiab-admin-console/assets

not a problem. this is the git repo and will be present as part of the install. it is also not used after the install.

I think we have come down to two strategies:

  1. mount an external drive LIBRARY-EXT on /library
  2. mount it on /library-ext and bind mount its internal library directory on /library

@jvonau says 2) is not viable because df will return the wrong or no size. If that is true then 1) should be used.

The only downside is that LIBRARY-EXT will be usable in Admin Console Manage Content as an internal device, but not as an external device mountable on another system.

@andrewclark5
Copy link
Author

Perhaps you could share your understanding of what is needed and the tools that you are going to use to support your endeavor. My guidance is recorded in #3106, 'b' is not a problem if you make both data and os 'images' at the same time from the source machine.

Ideally we will have 2 separate images for /library and OS/IIAB
But it may be that /library is just held on the big TT server and the file tree dragged over with rsync onto the target server.
It would be great if a script or program could be run from within Admin Console with the Source and Target locations user-specified to keep them away from command line disasters.
We will probably use Foxclone for imaging as it seems to work well. It can be booted from USB or possibly PXE over the network as we do with our wiping tools.
Ideally there will be a specific image for each different type of server although I have found in practice that I can get away with mis-matched image builds up to a point. I recently restored the image from a T620 thin client to a Lenovo X240 laptop and it worked first time. I was without an external USB mouse driver for the first 20 mins then it fixed itself!

@holta holta added the question label Jun 9, 2022
@holta holta added this to the 8.0 milestone Jun 9, 2022
@holta
Copy link
Member

holta commented Jun 9, 2022

Quick Recap from our weekly community call (http://minutes.iiab.io) ~8 hours ago.

@tim-moody's post from June 4th appears to be the most immediately useful here: (for this ticket especially)

To reconfirm that the various df issues are all under control, @shanti-bhardwa and @andrewclark5 will try this out to verify that it works well on Mint 20.3+ on the hardware most relevant for Turing Trust — verifying that it allows for:

  1. Easily mountable (easily interchangeable) external IIAB content disks for /library
  2. With Admin Console's (http://box.lan/admin) > Install Content > Manage Content also continuing to work, in support of reading/writing Content Packs etc to-and-from external USB drives.

@jvonau had suggestions regarding future use of a loopback file (loopback mount?) towards possibly streamlining /library content drives/directories/etc in future (much like a partition, but not quite the same). For example in cases where 100 or 128 GB of /library content has already been decided upon, and the inevitable last-minute content changes calm down, allowing this "content collection" [*] to be circulated among many similar IIAB's?

[*] Not quite a Learning Bouquet or "preset", but a very similar idea!

Finally, we did not have time to complete the discussion, as there are several quite subtle + nuanced issues to review regarding what belongs "externally" on /library — versus what belongs "internally" on IIAB root filesystems — to be discussed on a future call.

@andrewclark5
Copy link
Author

andrewclark5 commented Jun 10, 2022 via email

@tim-moody
Copy link
Contributor

instructions have gotten spread over so many locations that things easily go missing.

mount: /library-ext: mount point does not exist.
mount: /library: mount point does not exist.

as the error messages indicate you have skipped this step

mkdir /library
mkdir /library-ext

now mount -a should work

I take it that using an internal partition is just an experiment and not the eventually intended configuration.

@tim-moody
Copy link
Contributor

/dev/sda6 also needs a library directory on it before mount -a will work

@andrewclark5
Copy link
Author

andrewclark5 commented Jun 10, 2022 via email

@tim-moody
Copy link
Contributor

tim-moody commented Jun 10, 2022

if the data (library) partition is internal I would think it would not ever be mounted externally on another system

if that is the case then the bind mount is not really needed and you can mount /dev/sda6 directly on /library in fstab and comment out the other bind mount line

all of my nuc machines are configured like this

@tim-moody
Copy link
Contributor

I should add that the bind mount will still work even for an internal partition if standardization is the top priority.

@andrewclark5
Copy link
Author

andrewclark5 commented Jun 10, 2022 via email

@tim-moody
Copy link
Contributor

/dev/sda6 is now mounted, but looks as if I need to change it to /library
and not /library-ext, which is what is appearing in the File System?

So now it looks like you have unexpectedly switched strategies to mounting the internal partition directly on /library.

Let's get the main strategy working first.

with /dev/sda6 mounted on /library-ext
mkdir /library-ext/library

with fstab as

LABEL=IIAB-LIBRARY-EXT /library-ext ext4 defaults,nofail 0 0
/library-ext/library /library none defaults,bind,nofail 0 0

mount -a

@andrewclark5
Copy link
Author

andrewclark5 commented Jun 10, 2022 via email

@andrewclark5
Copy link
Author

andrewclark5 commented Jun 10, 2022 via email

@jvonau
Copy link
Contributor

jvonau commented Jun 10, 2022

Having to create /library-ext/library forces editing /etc/fsab and using mount -a twice adding the two /etc/fstab entries individually.
First edit to add the mounting of /library-ext line.
sudo mount -a
sudo mkdir /library-ext/library
Second edit to fstab adding the bindmount line.
sudo mount -a

I've baked this functionality into #3051

@andrewclark5
Copy link
Author

andrewclark5 commented Jun 10, 2022

Hi Jerry

That seems like a good explanation! I am talking to Tim on email just now as he is out so I see you have cc your text to him by email.

@andrewclark5
Copy link
Author

andrewclark5 commented Jun 10, 2022 via email

@jvonau
Copy link
Contributor

jvonau commented Jun 10, 2022

Should I delete all the superfluous folders and start again with fstab?

On the OS filesystem there should be two folders /library /library-ext
On the content filesystem there should be /library, that would be /library-ext/library when LABEL= is mounted at /library-ext

And what are we trying to achieve with the library folder structure?

Think Tim's goal is to avoid rewriting current code in admin-console to be able to extend the existing import/export functionality.

@andrewclark5
Copy link
Author

@tim-moody @jvonau @holta
Ah. I see. OK, I'm just going to edit fstab, delete the other unwanted folders and reboot and see what I have then.

@shanti-bhardwa
Copy link

shanti-bhardwa commented Jun 11, 2022

Done some testing on HP t620 with 16GB flash drive. Results look promising. Here is a summary.

  1. Built fresh Mint 20.3. from .iso file and installed it on t620. I am using Raspberry PI 2B USB WI-FI dongle as my WI-FI interface.
  2. Updated the system using sudo apt update & sudo apt upgrade to bring everything to current levels. Added Gparted as an addition utility.
  3. Created two folders on root with sudo mkdir /library and sudo mkdir /library-ext
  4. Formatted my external drive. This is a 32GB SSD in a SATA/USB caddy with label IIAB-LIBRARY-EXT with ext4 file format. I used these two commands
    sudo mkfs.ext4 /dev/sdb1 and sudo e2label /dev/sdb1 IIAB-LIBRARY-EXT. can also be done using Disks utility or from File Manager, but have not tested this.
  5. Installed the drive and then created a folder with sudo mkdir /library-ext/library on this USB device.
  6. Then added my /etc/fstab entries as follows. These two entries tell me that USB will always be mounted at /library-ext by mount at boot time & bind makes the two directory structures the same. IIAB puts everthing under /library so at IIAB build time only the /library-ext/library on USB is used as the real srorage,
    LABEL=IIAB-LIBRARY-EXT /library-ext ext4 defaults,nofail 0 0
    /library-ext/library /library none defaults,bind,nofail, 0 0
  7. Install IIAB 8.0. Install went without a hitch. The sudo iiab-diagnostics is here : http:/sprunge.us/InAh1R
  8. Downloaded few modules and worked fine.
    10 . Thinks I have noticed. df -h shows the usb drive mounted on /media/usb0 & in on admin console cannot get system storage.
  9. The machine is on vpn at 10.8.0.82. You can ssh user is sb01 and password is shanti

@shanti-bhardwa
Copy link

I think this going work really well. You can see the amount of storage on the uSB drives from Manage Content option.
My iiab-admin password is also shanti, so try adding some content if you want.
The external drive is hsown as mounted on /media/usb0. The second drive gets mounted on /media/usb1

@shanti-bhardwa
Copy link

Trying to get System Storage on Admin Console gives me this error
GET-STORAGE-INFO FAILED and reported Unexpected error in Command GET-STORAGE-INFO

@shanti-bhardwa
Copy link

Apologies, the machine is on vpn 10.8.0.62. My user id is sb01 with password shanti. Also iiab-admin password is shanti.
I will leave this machine connected for rest of the day.

@shanti-bhardwa
Copy link

Next steps for me to test is to create more External drives. So I will clone the IIAB-LIBRARY-EXT.
One drive will have Wikipedia , second Maps, third TED Video, fourth Kolibri Channels. See how it works

@shanti-bhardwa
Copy link

If you use gparted to look at the drives mounted then IIAB-LIBRARY-EXT which is /dev/sdb1 is mounted in three places /library, /library-ext and /media/usb0. You also get the used and unused storage size. So there must be other ways to get storage information without using df command.

@tim-moody
Copy link
Contributor

tim-moody commented Jun 11, 2022

df -h shows the usb drive mounted on /media/usb0

even after reboot? (leaving it attached)

GET-STORAGE-INFO FAILED

yes it's a bug - not compatible with df change for linux mint.

@andrewclark5
Copy link
Author

andrewclark5 commented Jun 11, 2022 via email

@andrewclark5
Copy link
Author

andrewclark5 commented Jun 11, 2022 via email

@jvonau
Copy link
Contributor

jvonau commented Jun 27, 2022

Sure. Will do both.

Should add the IIAB machine display is stable again. I hate random stuff like this!

Would you believe that the "clean" Mint-only T620-2 is now exhibiting the same blacking-out screen bug! Here is the output from .xsession-errors

Think this point to the use of 'oem' install or the video subsystem as the primary cause of the 'crash'

@andrewclark5
Copy link
Author

andrewclark5 commented Jun 27, 2022 via email

@tim-moody
Copy link
Contributor

I keep wondering if this is a timing problem: some cinnamon dependency not being available in time for auto login. Maybe more likely on IIAB as there are more services to start at boot.

@jvonau
Copy link
Contributor

jvonau commented Jun 27, 2022

Sure. Will do both.
Should add the IIAB machine display is stable again. I hate random stuff like this!

Would you believe that the "clean" Mint-only T620-2 is now exhibiting the same blacking-out screen bug! Here is the output from .xsession-errors

Think this point to the use of 'oem' install or the video subsystem as the primary cause of the 'crash'

Instructed to proceed through the oem phase which will prompt on next boot to create new user/password, this is going to be the shipping state of the device. After new user creation, no complaints about desktop glitches yet, left to run overnight.

user1@HP-t620-1:~$ cat .xsession-errors
dbus-update-activation-environment: setting DBUS_SESSION_BUS_ADDRESS=unix:path=/run/user/1000/bus
dbus-update-activation-environment: setting DISPLAY=:0
dbus-update-activation-environment: setting XAUTHORITY=/home/user1/.Xauthority
dbus-update-activation-environment: setting GTK_MODULES=gail:atk-bridge
dbus-update-activation-environment: setting QT_ACCESSIBILITY=1
dbus-update-activation-environment: setting SHELL=/bin/bash
dbus-update-activation-environment: setting QT_ACCESSIBILITY=1
dbus-update-activation-environment: setting XDG_CONFIG_DIRS=/etc/xdg/xdg-cinnamon:/etc/xdg
dbus-update-activation-environment: setting XDG_SESSION_PATH=/org/freedesktop/DisplayManager/Session0
dbus-update-activation-environment: setting LANGUAGE=en_GB:en
dbus-update-activation-environment: setting DESKTOP_SESSION=cinnamon
dbus-update-activation-environment: setting GTK_MODULES=gail:atk-bridge
dbus-update-activation-environment: setting PWD=/home/user1
dbus-update-activation-environment: setting XDG_SESSION_DESKTOP=cinnamon
dbus-update-activation-environment: setting LOGNAME=user1
dbus-update-activation-environment: setting XDG_SESSION_TYPE=x11
dbus-update-activation-environment: setting GPG_AGENT_INFO=/run/user/1000/gnupg/S.gpg-agent:0:1
dbus-update-activation-environment: setting XAUTHORITY=/home/user1/.Xauthority
dbus-update-activation-environment: setting XDG_GREETER_DATA_DIR=/var/lib/lightdm-data/user1
dbus-update-activation-environment: setting GDM_LANG=en_GB
dbus-update-activation-environment: setting HOME=/home/user1
dbus-update-activation-environment: setting IM_CONFIG_PHASE=1
dbus-update-activation-environment: setting LANG=en_GB.UTF-8
dbus-update-activation-environment: setting XDG_SEAT_PATH=/org/freedesktop/DisplayManager/Seat0
dbus-update-activation-environment: setting XDG_SESSION_CLASS=user
dbus-update-activation-environment: setting USER=user1
dbus-update-activation-environment: setting DISPLAY=:0
dbus-update-activation-environment: setting SHLVL=1
dbus-update-activation-environment: setting XDG_RUNTIME_DIR=/run/user/1000
dbus-update-activation-environment: setting GTK3_MODULES=xapp-gtk3-module
dbus-update-activation-environment: setting XDG_DATA_DIRS=/usr/share/cinnamon:/usr/share/gnome:/home/user1/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share
dbus-update-activation-environment: setting PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin
dbus-update-activation-environment: setting GDMSESSION=cinnamon
dbus-update-activation-environment: setting DBUS_SESSION_BUS_ADDRESS=unix:path=/run/user/1000/bus
dbus-update-activation-environment: setting _=/usr/bin/dbus-update-activation-environment
[cinnamon-settings-daemon-smartcard] Failed to start: no suitable smartcard driver could be found
** Message: 22:27:14.246: Existing monitor config (/home/user1/.config/cinnamon-monitors.xml) not found at startup. Looking for legacy configuration (monitors.xml)
[cinnamon-settings-daemon-smartcard] Failed to start: no suitable smartcard driver could be found
Cjs-Message: 22:27:18.929: Profiler is disabled. Not setting up signals.
Clutter-Message: 22:27:19.057: Sync method: PRESENTATION TIME
Cjs-Message: 22:27:20.955: JS LOG: About to start Cinnamon

(nm-applet:1983): libnotify-WARNING **: 22:27:22.244: Failed to connect to proxy

(nm-applet:1983): nm-applet-WARNING **: 22:27:22.276: Failed to show notification: GDBus.Error:org.freedesktop.DBus.Error.ServiceUnknown: The name org.freedesktop.Notifications was not provided by any .service files
Cjs-Message: 22:27:22.727: JS LOG: [LookingGlass/info] Cinnamon.AppSystem.get_default() started in 1714 ms
Cjs-Message: 22:27:22.840: JS LOG: [LookingGlass/info] loading user theme: /usr/share/themes/Mint-Y-Dark/cinnamon/cinnamon.css
Cjs-Message: 22:27:23.023: JS LOG: [LookingGlass/info] added icon directory: /usr/share/themes/Mint-Y-Dark/cinnamon
Cjs-Message: 22:27:24.631: JS LOG: [LookingGlass/info] loaded at Mon Jun 27 2022 22:27:24 GMT+0100 (British Summer Time)
Cjs-Message: 22:27:24.631: JS LOG: Cinnamon started at Mon Jun 27 2022 22:27:24 GMT+0100 (British Summer Time)
Cjs-Message: 22:27:24.799: JS LOG: [LookingGlass/info] ExtensionSystem started in 16 ms
Cjs-Message: 22:27:24.800: JS LOG: [LookingGlass/info] NaNms
Cjs-Message: 22:27:24.802: JS LOG: [LookingGlass/info] SearchProviderManager started in 8 ms
openGL version 4.5 detected (GL3 Cogl Driver)
MetaSyncRing disabled: couldn't find required GL extensions, or the minimum safe openGL version was not met
cinnamon-session[1677]: WARNING: t+12.29410s: Detected that screensaver has appeared on the bus
Cjs-Message: 22:27:25.688: JS LOG: [LookingGlass/info] Loaded applet show-desktop@cinnamon.org in 532 ms
Cjs-Message: 22:27:25.746: JS LOG: [LookingGlass/info] Role locked: tray
Cjs-Message: 22:27:25.793: JS LOG: [LookingGlass/info] Loaded applet systray@cinnamon.org in 100 ms
Cjs-Message: 22:27:25.960: JS LOG: [LookingGlass/info] Loaded applet removable-drives@cinnamon.org in 165 ms
Cjs-Message: 22:27:26.058: JS LOG: [LookingGlass/info] Loaded applet favorites@cinnamon.org in 98 ms
St-Message: 22:27:26.085: cogl npot texture sizes SUPPORTED
Cjs-Message: 22:27:26.159: JS LOG: [LookingGlass/info] Role locked: notifications
** Message: 22:27:26.277: nemo-desktop: session is cinnamon, establishing proxy
Cjs-Message: 22:27:26.283: JS LOG: [LookingGlass/info] Loaded applet notifications@cinnamon.org in 225 ms
Cjs-Message: 22:27:26.430: JS LOG: [LookingGlass/info] Loaded applet printers@cinnamon.org in 145 ms
Cjs-Message: 22:27:26.571: JS LOG: [LookingGlass/info] Loaded applet keyboard@cinnamon.org in 130 ms
Cjs-Message: 22:27:26.608: JS LOG: [LookingGlass/info] Loaded applet xapp-status@cinnamon.org in 36 ms
Cjs-Message: 22:27:26.994: JS LOG: [LookingGlass/info] Loaded applet calendar@cinnamon.org in 386 ms
Cjs-Message: 22:27:27.149: JS LOG: [LookingGlass/info] Role locked: panellauncher
Cjs-Message: 22:27:27.256: JS LOG: [LookingGlass/info] Loaded applet grouped-window-list@cinnamon.org in 261 ms
Cjs-Message: 22:27:27.328: JS LOG: [LookingGlass/info] Loaded applet power@cinnamon.org in 71 ms
Cjs-Message: 22:27:27.405: JS LOG: [LookingGlass/info] Adding XAppStatusIcon: nm-applet (:1.50/org/x/StatusIcon/Icon)

** (csd-power:1826): CRITICAL **: 22:27:27.746: abs_to_percentage: assertion 'max > min' failed
Cjs-Message: 22:27:28.058: JS LOG: [LookingGlass/info] Loaded applet sound@cinnamon.org in 730 ms
Cjs-Message: 22:27:28.274: JS LOG: [LookingGlass/info] Hiding XAppStatusIcon (we have an applet): nm-applet (:1.50/org/x/StatusIcon/Icon)
Cjs-Message: 22:27:28.362: JS LOG: [LookingGlass/info] Loaded applet network@cinnamon.org in 302 ms
Cjs-Message: 22:27:29.395: JS LOG: [LookingGlass/info] Loaded applet menu@cinnamon.org in 1033 ms
Cjs-Message: 22:27:29.399: JS LOG: [LookingGlass/info] AppletManager started in 4763 ms
Cjs-Message: 22:27:29.411: JS LOG: [LookingGlass/info] Cinnamon took 8455 ms to start
Cjs-Message: 22:27:29.499: JS LOG: Unknown network device type, is 14
Cjs-Message: 22:27:42.031: JS LOG: [LookingGlass/info] Adding XAppStatusIcon: mintUpdate.py (:1.77/org/x/StatusIcon/Icon)
cinnamon-session[1677]: WARNING: t+42.33862s: Detected that screensaver has left the bus
Cjs-Message: 22:28:01.337: JS LOG: [LookingGlass/info] Adding XAppStatusIcon: mintreport (:1.87/org/x/StatusIcon/Icon)

(polkit-gnome-authentication-agent-1:1987): polkit-gnome-1-WARNING **: 22:33:16.171: Couldn't open user icon: Failed to open file “/home/iiab-admin/.face”: No such file or directory

(polkit-gnome-authentication-agent-1:1987): polkit-gnome-1-WARNING **: 22:33:16.261: Couldn't open user icon: Failed to open file “/home/user1/.face”: No such file or directory

(polkit-gnome-authentication-agent-1:1987): polkit-gnome-1-WARNING **: 22:33:22.231: Couldn't open user icon: Failed to open file “/home/iiab-admin/.face”: No such file or directory

(polkit-gnome-authentication-agent-1:1987): polkit-gnome-1-WARNING **: 22:33:22.252: Couldn't open user icon: Failed to open file “/home/user1/.face”: No such file or directory
Cjs-Message: 22:33:27.519: JS LOG: Unknown network device type, is 16
cinnamon-session[1677]: WARNING: t+1511.64363s: Detected that screensaver has appeared on the bus

@jvonau
Copy link
Contributor

jvonau commented Jun 28, 2022

Seem ok poking around but kolibri's cpu usage seems a bit high. Is that 'out of the box' or are there channels loaded?

   PID USER      PR  NI    VIRT    RES    SHR S  %CPU  %MEM     TIME+ COMMAND                                                 
   1424 kolibri   20   0 2079496  82412   7156 S  16.9   2.4 103:03.41 kolibri                                                 
    794 root      20   0  773856 115236  77736 S   1.0   3.4   1:47.87 Xorg                                                    
  11406 user1     20   0   11888   3924   3420 R   1.0   0.1   0:00.11 top                                                     
   3773 user1     20   0  806860  66728  46220 S   0.7   1.9   0:17.89 cinnamon-screen                                         
    828 mysql     20   0 1710492  78504  18592 S   0.3   2.3   1:06.55 mysqld                                                  
   1025 root      20   0 1202568  42968   5424 S   0.3   1.2   1:20.85 kalite                                                  
   1953 user1     20   0 3773524 176176 107088 S   0.3   5.1   2:43.00 cinnamon

@andrewclark5
Copy link
Author

andrewclark5 commented Jun 28, 2022 via email

@andrewclark5
Copy link
Author

I just rebooted and checked the running processes again. It does look as if Kolibri is a bit resouce-hungry as there are no clients running IIAB.

top - 09:20:50 up 9 min, 1 user, load average: 1.18, 1.34, 0.84
Tasks: 204 total, 1 running, 203 sleeping, 0 stopped, 0 zombie
%Cpu(s): 13.4 us, 3.7 sy, 0.0 ni, 82.7 id, 0.2 wa, 0.0 hi, 0.0 si, 0.0 st
MiB Mem : 3357.9 total, 838.6 free, 1316.1 used, 1203.2 buff/cache
MiB Swap: 683.5 total, 683.5 free, 0.0 used. 1790.3 avail Mem

    PID USER      PR  NI    VIRT    RES    SHR S  %CPU  %MEM     TIME+ COMMAND                                  
   1695 kolibri   20   0 2079728  82096   7112 S  16.9   2.4   1:22.34 kolibri                                  
   1700 user1     20   0 3773492 176920 108160 S   4.2   5.1   0:26.91 cinnamon                                 
    805 root      20   0  698544 104580  68832 S   3.6   3.0   0:17.86 Xorg                                     
   2201 user1     20   0  464620  40136  30940 S   3.6   1.2   0:03.82 gnome-terminal-                          
   2224 user1     20   0   11.0g 372688 203620 S   2.3  10.8   1:31.27 firefox-bin                              
   2525 user1     20   0   12032   3948   3304 R   1.0   0.1   0:01.97 top                                      
   2333 user1     20   0 2436496 108912  77832 S   0.7   3.2   0:10.27 Privileged Cont                          
    290 root      20   0       0      0      0 I   0.3   0.0   0:03.63 kworker/0:3-events                       
   1020 root      20   0 1202576  43024   5452 S   0.3   1.3   0:01.81 kalite                                   
   1537 user1     20   0  299484  25084  19056 S   0.3   0.7   0:00.46 csd-keyboard                             
   2438 user1     20   0 2468976 131756  82092 S   0.3   3.8   0:07.09 Isolated Web Co                          
   2454 user1     20   0 2682692 181276 102816 S   0.3   5.3   0:27.52 Isolated Web Co                          
   2589 root      20   0       0      0      0 I   0.3   0.0   0:00.45 kworker/1:0-events                       
      1 root      20   0  168216  11760   8528 S   0.0   0.3   0:03.04 systemd                                  
      2 root      20   0       0      0      0 S   0.0   0.0   0:00.00 kthreadd                                 
      3 root       0 -20       0      0      0 I   0.0   0.0   0:00.00 rcu_gp                                   
      4 root       0 -20       0      0      0 I   0.0   0.0   0:00.00 rcu_par_gp                               
      6 root       0 -20       0      0      0 I   0.0   0.0   0:00.00 kworker/0:0H-kblockd                     
      7 root      20   0       0      0      0 I   0.0   0.0   0:00.19 kworker/u8:0-events_power_efficient      
      8 root       0 -20       0      0      0 I   0.0   0.0   0:00.00 mm_percpu_wq                             
      9 root      20   0       0      0      0 S   0.0   0.0   0:00.15 ksoftirqd/0                              
     10 root      20   0       0      0      0 I   0.0   0.0   0:01.02 rcu_sched                                
     11 root      rt   0       0      0      0 S   0.0   0.0   0:00.00 migration/0                              
     12 root     -51   0       0      0      0 S   0.0   0.0   0:00.00 idle_inject/0                            
     14 root      20   0       0      0      0 S   0.0   0.0   0:00.00 cpuhp/0                                  
     15 root      20   0       0      0      0 S   0.0   0.0   0:00.00 cpuhp/1                                  
     16 root     -51   0       0      0      0 S   0.0   0.0   0:00.00 idle_inject/1                            
     17 root      rt   0       0      0      0 S   0.0   0.0   0:00.29 migration/1                              
     18 root      20   0       0      0      0 S   0.0   0.0   0:00.18 ksoftirqd/1                              

@holta
Copy link
Member

holta commented Jun 28, 2022

I just rebooted and checked the running processes again. It does look as if Kolibri is a bit resouce-hungry as there are no clients running IIAB.

Definitely. This is a known issue. Kolibri uses significant resources even when idle.

While this doesn't break the bank, certainly there are many who find this aggravating, and it's something to be aware of (-:

@andrewclark5
Copy link
Author

andrewclark5 commented Jun 28, 2022 via email

@andrewclark5
Copy link
Author

OK I have been hammering this machine all day with reboots and downloads of RACHEL modules without a glitch. It seems @jvonau Jerry was probably right and it was the OEM build option that pushed it over the edge! We may need to be careful in future about adding any further load to the desktop system as I deduce that it might be a bit fragile in some respects.

I have made an image of the working system as-is, with both auto-mount and auto-login disabled, and will make it available to @shanti-bhardwa for testing.

I am sorting out my Library options fixing any that don't work and downloading a huge 40GB Wikihow module to add to our biggest library.

Many thanks for your diagnostic efforts guys!

@jvonau
Copy link
Contributor

jvonau commented Jun 30, 2022

I'll be waiting for my knighthood from her majesty the queen for this work to expand the commonwealth. ;)

@andrewclark5
Copy link
Author

Arise, Sir Jerry! (Well, I'll support your case to Her Maj anyway). A.

@jvonau
Copy link
Contributor

jvonau commented Sep 18, 2022

notes on fuse and gvfs

@andrewclark5
Copy link
Author

andrewclark5 commented Oct 11, 2022 via email

@holta
Copy link
Member

holta commented Oct 11, 2022

Should I go ahead now and install IIAB?

Please! As explained at https://download.iiab.io, IIAB install instructions have been shortened to:

curl iiab.io/install.txt | bash

[1] There were significant resilience improvements over the past month:
https://github.com/iiab/iiab/milestone/8?closed=1

[2] Just FYI in coming days (or coming weeks, depending on Kiwix's decision) kiwix-tools 3.3.1 should released and included as a part of IIAB, somewhat enhancing how ZIM files work (like Wikipedia, etc), potentially also improving search:
https://github.com/kiwix/kiwix-tools/blob/master/Changelog

@andrewclark5
Copy link
Author

andrewclark5 commented Oct 11, 2022 via email

@holta
Copy link
Member

holta commented Oct 12, 2022

Just FYI @andrewclark5 @shanti-bhardwa:

  1. A small fix was just merged to allow for Sugarizer on Mint 21

  2. If your IIAB machine's internal WiFi should happen to be blocked, here's a general tip (manual for now) you might already know about:

@holta
Copy link
Member

holta commented Oct 12, 2022

Note there is a small typo in your fstab syntax just in case we cut and paste it LABEL=IIAB-LIBRART-EXT

  1. Typo fixed in @shanti-bhardwa's above 2022-06-11 tips:

    Best strategy for preparing multiple IIAB servers using imaging #3247 (comment)

  2. If there are very concise/precise suggestions as to what's missing in http://FAQ.IIAB.IO -> "41 Can I permanently attach an external USB drive, to add more content?" please LMK?

    https://wiki.iiab.io/go/FAQ#Can_I_permanently_attach_an_external_USB_drive,_to_add_more_content%3F

    Thanks All!

@jvonau
Copy link
Contributor

jvonau commented Oct 13, 2022

2. If there are very concise/precise suggestions as to what missing in http://FAQ.IIAB.IO -> "41 Can I permanently attach an external USB drive, to add more content?" please LMK?
   https://wiki.iiab.io/go/FAQ#Can_I_permanently_attach_an_external_USB_drive,_to_add_more_content%3F

The device should be partitioned:

mkfs.ext4 /dev/sdX
should be /dev/sdXn

e2label /dev/sdX IIAB-LIBRARY-EXT
should be /dev/sdXn

@holta
Copy link
Member

holta commented Oct 13, 2022

mkfs.ext4 /dev/sdX
should be /dev/sdXn

The original instructions intentionally allowed for X to be a single character or multiple characters e.g. if the USB drive is partitioned:

Before running the above, change 'X' to correspond to your actual USB drive. You can uncover this (its device name) by running commands like lsblk or blkid with df -h.

I've reworked the FAQ instructions a bit to try to make that (more) explicitly clear.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

5 participants