Kol user [KOLIBRI_HOME and KOLIBRI_USER best practices needed, hopefully with help from Learning Equality] #3505

jvonau · 2023-03-23T15:03:05Z

Fixes bug:

issues raised in #3504

Description of changes proposed in this pull request:

kalite approach to dealing with HOME and USER

Smoke-tested on which OS or OS's:

U-22.04 VM

Mention a team member @username e.g. to help with code review:

jvonau · 2023-03-23T15:22:01Z

ubuntu@box:/opt/iiab/iiab$ /usr/local/bin/kolibri

Please run: sudo kolibri

ubuntu@box:/opt/iiab/iiab$ sudo /usr/local/bin/kolibri
INFO: No C extensions are available for this platform
/usr/lib/python3/dist-packages/pkg_resources/__init__.py:116: PkgResourcesDeprecationWarning: 1.1build1 is an invalid version and will not be supported in a future release
  warnings.warn(
/usr/lib/python3/dist-packages/pkg_resources/__init__.py:116: PkgResourcesDeprecationWarning: 0.1.43ubuntu1 is an invalid version and will not be supported in a future release
  warnings.warn(
Usage: kolibri [OPTIONS] COMMAND [ARGS]...

  Kolibri command-line utility

  Details for each main command: kolibri COMMAND --help

  List of additional management commands: kolibri manage help

  For more information, see: https://kolibri.readthedocs.io/

Options:
  --version  Show the version and exit.
  --help     Show this message and exit.

Commands:
  configure  Configure Kolibri and enabled plugins
  manage     Django management commands.
  plugin     Manage Kolibri plugins
  restart    Restart the Kolibri process
  services   Start worker processes
  shell      Launch a Django shell
  start      Start the Kolibri process
  status     Show the status of the Kolibri process
  stop       Stop the Kolibri process

jvonau · 2023-03-23T15:27:48Z

Not quite there yet 'command --help' doesn't work.

holta · 2023-03-23T15:38:50Z

Changing kolibri's home directory in /etc/passwd to /library/kolibri is probably wise (if only for doc and hygiene purposes).
However there are questions as to how much more is truly necessary and/or imposes long-term maintenance that can be avoided.

One Example: $@ is usually considered unsafe, (in favor of "$@").

Better to avoid all such complexity (whenever possible anyway).

When ongoing/eternal maintenance can be hopefully avoided — whether or not Kolibri upstream behavior (with regards to vars like KOLIBRI_HOME and KOLIBRI_USER) changes next month or next year or next decade — all of which are very real possibilities ⛈️

jvonau · 2023-03-23T15:40:52Z

then fix it yourself

jvonau · 2023-03-23T15:59:46Z

A little FYI something is messed up if any user other than could run /usr/bin/kolibri

ubuntu@box:/opt/iiab/iiab$ /usr/bin/kolibri manage help
INFO: No C extensions are available for this platform
/usr/lib/python3/dist-packages/pkg_resources/__init__.py:116: PkgResourcesDeprecationWarning: 1.1build1 is an invalid version and will not be supported in a future release
  warnings.warn(
/usr/lib/python3/dist-packages/pkg_resources/__init__.py:116: PkgResourcesDeprecationWarning: 0.1.43ubuntu1 is an invalid version and will not be supported in a future release
  warnings.warn(
INFO     2023-03-23 10:51:12,958 Option DEBUG in section [Server] being overridden by environment variable KOLIBRI_DEBUG
INFO     2023-03-23 10:51:12,958 Option DEBUG_LOG_DATABASE in section [Server] being overridden by environment variable KOLIBRI_DEBUG_LOG_DATABASE
INFO     2023-03-23 10:51:12,962 Option DEBUG in section [Server] being overridden by environment variable KOLIBRI_DEBUG
INFO     2023-03-23 10:51:12,962 Option DEBUG_LOG_DATABASE in section [Server] being overridden by environment variable KOLIBRI_DEBUG_LOG_DATABASE
INFO     2023-03-23 10:51:12,964 Attempting to setup using pre-migrated databases
INFO     2023-03-23 10:51:12,964 Copied preseeded database from /usr/lib/python3/dist-packages/kolibri/dist/home/db.sqlite3 to /home/ubuntu/.kolibri/db.sqlite3
INFO     2023-03-23 10:51:12,965 Copied preseeded database from /usr/lib/python3/dist-packages/kolibri/dist/home/syncqueue.sqlite3 to /home/ubuntu/.kolibri/syncqueue.sqlite3
INFO     2023-03-23 10:51:12,965 Copied preseeded database from /usr/lib/python3/dist-packages/kolibri/dist/home/networklocation.sqlite3 to /home/ubuntu/.kolibri/networklocation.sqlite3
INFO     2023-03-23 10:51:12,965 Copied preseeded database from /usr/lib/python3/dist-packages/kolibri/dist/home/notifications.sqlite3 to /home/ubuntu/.kolibri/notifications.sqlite3
INFO     2023-03-23 10:51:13,086 Running Kolibri with the following settings: kolibri.deployment.default.settings.base
INFO     2023-03-23 10:51:13,490 New install, version: 0.15.12
INFO     2023-03-23 10:51:13,491 Running update routines for new version...
Operations to perform:
  Apply all migrations: admin, analytics, auth, bookmarks, content, contenttypes, device, discovery, exams, kolibriauth, lessons, logger, morango, notifications, sessions
Running migrations:
  No migrations to apply.
Operations to perform:
  Apply all migrations: admin, analytics, auth, bookmarks, content, contenttypes, device, discovery, exams, kolibriauth, lessons, logger, morango, notifications, sessions
Running migrations:
  No migrations to apply.
Operations to perform:
  Apply all migrations: admin, analytics, auth, bookmarks, content, contenttypes, device, discovery, exams, kolibriauth, lessons, logger, morango, notifications, sessions
Running migrations:
  No migrations to apply.
Operations to perform:
  Apply all migrations: admin, analytics, auth, bookmarks, content, contenttypes, device, discovery, exams, kolibriauth, lessons, logger, morango, notifications, sessions
Running migrations:
  No migrations to apply.
Installed 2 object(s) from 1 fixture(s)
INFO     2023-03-23 10:51:14,002 Detected updates to plugins: kolibri.plugins.setup_wizard, kolibri.plugins.slideshow_viewer, kolibri.plugins.perseus_viewer, kolibri.plugins.user_profile, kolibri.plugins.facility, kolibri.plugins.media_player, kolibri.plugins.coach, kolibri.plugins.html5_viewer, kolibri.plugins.pdf_viewer, kolibri.plugins.epub_viewer, kolibri.plugins.user_auth, kolibri.plugins.default_theme, kolibri.plugins.learn, kolibri.plugins.device
INFO     2023-03-23 10:51:14,006 Running installation routines for kolibri.plugins.setup_wizard, installing 0.15.12
INFO     2023-03-23 10:51:14,006 kolibri.plugins.setup_wizard successfully updated
INFO     2023-03-23 10:51:14,007 Running installation routines for kolibri.plugins.slideshow_viewer, installing 0.15.12
INFO     2023-03-23 10:51:14,007 kolibri.plugins.slideshow_viewer successfully updated
INFO     2023-03-23 10:51:14,007 Running installation routines for kolibri.plugins.perseus_viewer, installing 0.15.12
INFO     2023-03-23 10:51:14,007 kolibri.plugins.perseus_viewer successfully updated
INFO     2023-03-23 10:51:14,007 Running installation routines for kolibri.plugins.user_profile, installing 0.15.12
INFO     2023-03-23 10:51:14,007 kolibri.plugins.user_profile successfully updated
INFO     2023-03-23 10:51:14,008 Running installation routines for kolibri.plugins.facility, installing 0.15.12
INFO     2023-03-23 10:51:14,008 kolibri.plugins.facility successfully updated
INFO     2023-03-23 10:51:14,008 Running installation routines for kolibri.plugins.media_player, installing 0.15.12
INFO     2023-03-23 10:51:14,008 kolibri.plugins.media_player successfully updated
INFO     2023-03-23 10:51:14,008 Running installation routines for kolibri.plugins.coach, installing 0.15.12
INFO     2023-03-23 10:51:14,008 kolibri.plugins.coach successfully updated
INFO     2023-03-23 10:51:14,008 Running installation routines for kolibri.plugins.html5_viewer, installing 0.15.12
INFO     2023-03-23 10:51:14,008 kolibri.plugins.html5_viewer successfully updated
INFO     2023-03-23 10:51:14,009 Running installation routines for kolibri.plugins.pdf_viewer, installing 0.15.12
INFO     2023-03-23 10:51:14,009 kolibri.plugins.pdf_viewer successfully updated
INFO     2023-03-23 10:51:14,009 Running installation routines for kolibri.plugins.epub_viewer, installing 0.15.12
INFO     2023-03-23 10:51:14,009 kolibri.plugins.epub_viewer successfully updated
INFO     2023-03-23 10:51:14,009 Running installation routines for kolibri.plugins.user_auth, installing 0.15.12
INFO     2023-03-23 10:51:14,009 kolibri.plugins.user_auth successfully updated
INFO     2023-03-23 10:51:14,009 Running installation routines for kolibri.plugins.default_theme, installing 0.15.12
INFO     2023-03-23 10:51:14,009 kolibri.plugins.default_theme successfully updated
INFO     2023-03-23 10:51:14,009 Running installation routines for kolibri.plugins.learn, installing 0.15.12
INFO     2023-03-23 10:51:14,009 kolibri.plugins.learn successfully updated
INFO     2023-03-23 10:51:14,010 Running installation routines for kolibri.plugins.device, installing 0.15.12
INFO     2023-03-23 10:51:14,010 kolibri.plugins.device successfully updated
INFO     2023-03-23 10:51:14,112 Importing 'tasks' module from django apps
INFO     2023-03-23 10:51:14,125 Invoking command help

Type 'kolibri manage help <subcommand>' for help on a specific subcommand.

Available subcommands:

[analytics]
    benchmark
    ping
    profile

[auth]
    bulkexportusers
    bulkimportusers
    changepassword
    createsuperuser
    deletefacility
    deleteuser
    deprovision
    exportusers
    flushsyncsessions
    fullfacilitysync
    importusers
    recreatefacility
    registerfacility
    resumesync
    sync
    user_info

[content]
    content
    deletechannel
    deletecontent
    exportchannel
    exportcontent
    generate_schema
    importchannel
    importcontent
    labeltestdata
    listchannels
    scanforcontent
    setchannelposition

[contenttypes]
    remove_stale_contenttypes

[device]
    provisiondevice

[deviceadmin]
    dbbackup
    dbrestore
    vacuum

[discovery]
    enumeratedrives

[django]
    check
    compilemessages
    createcachetable
    dbshell
    diffsettings
    dumpdata
    flush
    inspectdb
    loaddata
    makemessages
    makemigrations
    migrate
    sendtestemail
    shell
    showmigrations
    sqlflush
    sqlmigrate
    sqlsequencereset
    squashmigrations
    startapp
    startproject
    test
    testserver

[django_js_reverse]
    collectstatic_js_reverse

[logger]
    exportlogs
    generateuserdata

[morango]
    cleanupsyncs

[rest_framework]
    generateschema

[sessions]
    clearsessions

[staticfiles]
    collectstatic
    findstatic
    runserver

[tasks]
    base

Note the invoking user's home directory is being populated, ignoring what is in the config files, all with just calling 'help', Good luck.

holta · 2023-03-23T16:07:29Z

If we do move ahead with /etc/password hygiene (likely, can't hurt) using home: "{{ kolibri_home }}", note that create_home: no should probably also be preserved.

jvonau · 2023-03-23T16:16:24Z

not my issue have fun

holta · 2023-03-23T16:28:36Z

[Kolibri is] ignoring what is in [its own] config files

ubuntu@box:/opt/iiab/iiab$ /usr/bin/kolibri manage help

INFO 2023-03-23 10:51:12,964 Copied preseeded database from /usr/lib/python3/dist-packages/kolibri/dist/home/db.sqlite3 to /home/ubuntu/.kolibri/db.sqlite3

Yes.

RECAP: Kolibri documentation remains missing with a broken link[1] as to where things like KOLIBRI_HOME and KOLIBRI_USER should properly be set.

Leading to these ongoing problems.

Hopefully that will be cleared up in coming months with help from @jredrejo, @benjaoming or @radinamatic.

[1] Broken link https://kolibri.readthedocs.io/en/latest/advanced.html within https://github.com/learningequality/kolibri-installer-debian/blob/fdb96a173efff92d79789140201ac3f385b20364/debian/README.etc#L19 as discussed on learningequality/kolibri-installer-debian#115

tim-moody · 2023-03-23T21:27:19Z

to be clear, I will use python, so the current kolibri user with no home, shell, or password is not a problem.

jvonau · 2023-03-24T14:34:02Z

If you export KOLIBRI_HOME=/library/kolibri before /usr/bin/kolibri manage help
You get a traceback because the directory is owned by 'kolibri' and the user at the cmdline was 'ubuntu'

INFO: No C extensions are available for this platform
Traceback (most recent call last):
  File "/usr/bin/kolibri", line 11, in <module>
    load_entry_point('kolibri==0.15.12', 'console_scripts', 'kolibri')()
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 479, in load_entry_point
    return get_distribution(dist).load_entry_point(group, name)
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 2861, in load_entry_point
    return ep.load()
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 2465, in load
    return self.resolve()
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 2471, in resolve
    module = __import__(self.module_name, fromlist=['__name__'], level=0)
  File "/usr/lib/python3/dist-packages/kolibri/utils/cli.py", line 17, in <module>
    from kolibri.plugins import config
  File "/usr/lib/python3/dist-packages/kolibri/plugins/__init__.py", line 16, in <module>
    from kolibri.utils.conf import KOLIBRI_HOME
  File "/usr/lib/python3/dist-packages/kolibri/utils/conf.py", line 47, in <module>
    os.mkdir(LOG_ROOT)
PermissionError: [Errno 13] Permission denied: '/library/kolibri/logs'

However there is function called check_debian_user imported in cli,py that might of caught the other issue above and raised a warning/prompt, makes me wonder if this is an import ordering issue, should check if the user is valid before trying to modify the filesystem. Perhaps it's just an ownership issue of /library/kolibri/. Again good luck.

holta · 2023-03-24T15:05:45Z

function called check_debian_user imported in [cli.py]

Thanks much for highlighting that and your summary here:

kolibri manage runs on wrong directory #3504 (comment)

Until Kolibri clarifies "official" guidelines — as to where KOLIBRI_USER and KOLIBRI_HOME should best be set — ideally with SSOT / DRY in mind to help us all reduce brittleness in coming years 🙏

jvonau added 2 commits March 23, 2023 08:57

kolibri home directory

d568991

helper script

6733c86

holta added this to the 8.1 milestone Mar 23, 2023

holta added the question label Mar 23, 2023

touchups

a5cd47a

jvonau closed this Mar 23, 2023

jvonau deleted the kol_user branch March 23, 2023 15:42

holta changed the title ~~Kol user~~ Kol user [need to disambiguate KOLIBRI_HOME and KOLIBRI_USER, hopefully with help from Learning Equality] Mar 23, 2023

holta changed the title ~~Kol user [need to disambiguate KOLIBRI_HOME and KOLIBRI_USER, hopefully with help from Learning Equality]~~ Kol user [KOLIBRI_HOME and KOLIBRI_USER best practices needed, hopefully with help from Learning Equality] Mar 23, 2023

holta mentioned this pull request Mar 23, 2023

kolibri global vars #3506

Closed

holta mentioned this pull request Mar 24, 2023

Move 2 vars kolibri/defaults to default_vars.yml + Cleaner kolibri homedir in /etc/passwd #3507

Merged

jvonau mentioned this pull request Mar 24, 2023

kolibri manage runs on wrong directory #3504

Open

holta mentioned this pull request Mar 27, 2023

kolibri/tasks/install.yml spring cleaning (tighten up code, flow, in-line docs) #3514

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Kol user [KOLIBRI_HOME and KOLIBRI_USER best practices needed, hopefully with help from Learning Equality] #3505

Kol user [KOLIBRI_HOME and KOLIBRI_USER best practices needed, hopefully with help from Learning Equality] #3505

jvonau commented Mar 23, 2023

jvonau commented Mar 23, 2023

jvonau commented Mar 23, 2023

holta commented Mar 23, 2023 •

edited

jvonau commented Mar 23, 2023

jvonau commented Mar 23, 2023

holta commented Mar 23, 2023

jvonau commented Mar 23, 2023

holta commented Mar 23, 2023

tim-moody commented Mar 23, 2023

jvonau commented Mar 24, 2023

holta commented Mar 24, 2023

Kol user [KOLIBRI_HOME and KOLIBRI_USER best practices needed, hopefully with help from Learning Equality] #3505

Kol user [KOLIBRI_HOME and KOLIBRI_USER best practices needed, hopefully with help from Learning Equality] #3505

Conversation

jvonau commented Mar 23, 2023

Fixes bug:

Description of changes proposed in this pull request:

Smoke-tested on which OS or OS's:

Mention a team member @username e.g. to help with code review:

jvonau commented Mar 23, 2023

jvonau commented Mar 23, 2023

holta commented Mar 23, 2023 • edited

jvonau commented Mar 23, 2023

jvonau commented Mar 23, 2023

holta commented Mar 23, 2023

jvonau commented Mar 23, 2023

holta commented Mar 23, 2023

tim-moody commented Mar 23, 2023

jvonau commented Mar 24, 2023

holta commented Mar 24, 2023

holta commented Mar 23, 2023 •

edited