Komodo v1.18.3

Changelog

• Quick fix for an issue found shortly after releasing 1.18.2
  • Builds were failing when after changing the repo branch due to change in 1.18.2
  • This adds a git fetch call before the branch checkout to make sure all latest remote branches are known, fixing the issue.
• User Group: Allow admin to add Attach permission to Repo / Build from the UI

Komodo v1.18.2

Changelog

The two main features of this release are Linked Repos and Alert Maintenance Windows.

Repo Linking

For Repo linking, this means you can link Repos that you configure once in Komodo to multiple resources. This avoids having to configure your repo provider, name, and account multiple times (you can just select it in the dropdown), and also keeps things in sync if you change the underlying repo provider, name, branch, or account. This is available on all Resources that use git repos (Stacks, Builds, and Resource Syncs).

For Stacks, this means that multiple Stacks attached to the same Repo will share the same repo clone on individual hosts. You can migrate your Stacks to used the linked Repo in place, just note that it won't deal with any data mounted inside the clone folders. Those would need to be moved to the new Repo clone path (as specified in the Repo config). Ideally all your data mounts are outside of the repo. Config files which are committed inside the repo are fine to keep mounted in using relative path however.

Note that you can but don't need to attach a Server / Builder to the Repo. If you do configure an attachment on the Repo, the Repo can still be attached to eg Stacks on other Servers. The Repo server / builder attachments aren't considered by Resources depending on the Repo.

Alert Maintenance Windows

These allow you to configure time periods to suppress alerts when downtime is expected, such as for system updates. You can schedule either Daily, Weekly, or One Time maintenance windows, and configure them on individual Servers and Alerters.

Resource

• Server: Add docker lists search re #548
• Server: Add Periphery version indicator to Server tables
• Deployment / All containers: Support capitals and . in the names re #563
• Actions: Add komodo.execute_container_exec, komodo.execute_deployment_exec, komodo.execute_stack_exec methods
  • Similar to komodo.execute_terminal, but set up to connect directly to container exec
  • Only requires disable_container_exec = false in Periphery config. Direct terminals can be disabled.
• Stack, Build, Resource Sync: Can now link Komodo Repos (in the sidebar) directly to these resources
  • Stacks: Have them share a single repo clone on your hosts.
  • Don't need to attach Repo to any Server / Builder, but you can if you would like, to manually pull / reclone.
  • Note: Non admin users need Attach permission on the Repo to be able to link Resources to it.
• Server / Alerter: Add per-server and per-alerter maintenance window configuration to suppress alerts during planned time periods. #550 by @R3D2 and @mbecker20

Misc

• Failed OIDC provider initialization just error logs, doesn't crash core. Can still login with other providers. Re. #567
• OIDC login gets username using the standard userinfo endpoint, instead of token claims. Does not affect already created users in Komodo, only new users. Re. #557

Komodo v1.18.1

Changelog

Schedules Page

• Added a dedicated Schedules page to provide an overview of all your scheduled actions.

Server Alerting

• Server health alerts will now only be opened after 2 consecutive out-of-bounds conditions.
  • This helps a lot to reduce noise, like intermittent unreachable alerts and short CPU usage spikes
  • If you turned off alerts to reduce noise, you can try turning them back on and see how it is now.

Commits

• Commits made by Komodo no longer use --force in the push command.
  • This should prevent any more issues like #501
  • If you run into issues pushing, it depends on the resource type.
    • For Stacks, can temporarily enable the Reclone option, and Deploy. Afterwards can try commit again.
    • For Syncs / Builds, you should be able to use the refresh button in header and try again. Otherwise you can recreate the repo cache volume.

Resources

• Stack / Build / Repo / Sync: Add url links for attached repos in the resource headers / tables.
  • Github only: links to correct branch
• Stack / Deployment: Add dedicated "Deploying" state to improve the feedback
• Server: Add Core / Periphery version mismatch indicator
• Repo: Fix "On Pull" command during PullRepo operation re #580
• Stack: Run "Pre Deploy" command before docker compose config in #584 by @undaunt
  • Allows for easier integrations re #324

UI

• Clean up headers / quick links
• Improve frontend loading time re dependency loading
• Move Builders / Alerters out of sidebar and into settings.

Komodo v1.18.0

Changelog

🚨 This release moves official support to FerretDB v2. Users who deployed v1.17.5 or before using Postgres / Sqlite option are using FerretDB v1 and should eventually migrate using the FerretDB v2 Update Guide. Note that this is not a change to Komodo itself, only to the list of supported Mongo stand-ins. Users can update to 1.18.0 and continue to use FerretDB v1 if they wish.

🚨 Admins managing user permissions may need to modify the user access rules. In particular, container logs, docker inspect on containers, and terminal access are now gated behind additional permissions (for non admin users).

Specific Permissions

The main purpose of this release is to refine the access control / permissions system in Komodo. In 1.17.5 and before, access to resources was controlled only via access level (Read, Execute, Write). These levels provide access to the associated /read, /execute, and /write methods on resources, and it worked pretty well to provide RBAC.

Now with more potentially sensitive features, this is not quite enough to provide granular access control. To address this, specific permissions have been introduced in addition to Read, Execute, and Write levels.

• Logs: User can retrieve docker / docker compose logs on the associated resource.
  • Valid on Server, Stack, Deployment.
  • For admins wanting this permission by default for all users with read permissions, see below on default user groups.
• Inspect: User can "inspect" docker containers.
  • Valid on Server, Stack, Deployment.
  • On Servers: Access to this api will expose all container environments on the given server,
    and can easily lead to secrets being leaked to unintended users if not protected.
• Terminal: User can access the associated resource's terminal.
  • If given on a Server, this allows server level terminal access, and all container exec priviledges (Including attached Stacks / Deployments).
  • If given on a Stack or Deployment, this allows container exec terminal (even without Terminal on Server).
• Attach: User can "attach" other resources to the resource.
  • If given on a Server, allows users to attach Stacks, Deployments, Repos, and Builders.
  • If given on Build, allows users to attach the Build to Deployments
  • If given on a Builder, allows users to attach Builds.
• Processes: User can retrieve the full running process list on the Server.

The above specific permissions are defined in a list alongside their level. This list is open for future expansion / and the associated implementations may be refined in future releases as well. The list is also given here: https://komo.do/docs/permissioning#specific-permissions.

Default User Groups

Sometimes you will want to set a "baseline" set of permissions that all users will have on the Komodo instance. Previously this could only be done in very barebones way, by setting KOMODO_TRANSPARENT_MODE=true on the Komodo Core container. This would give all users a base level of "Read" on all resources.

In addition to the above permissions features, this release also adds an everyone mode to User Groups. If you enable this mode on a User Group, then all users will inherit those permissions as a base.

TOML Examples

As before, you are able to manage User Groups in Resource Syncs.

# Can define default rules in the Everyone group
[[user_group]]
name = "Everyone"
everyone = true
 # Can see servers, but no Logs / Inspect / Terminal permission
all.Server = "Read"
# This doesn't elevate specific stacks from None permissions,
# but if the user gets greater than Read from another permission,
# they will inherit the specific permissions
all.Stack = { level = "None", specific = ["Inspect", "Logs", "Terminal"] }
all.Deployment = { level = "None", specific = ["Inspect", "Logs", "Terminal"] }
# Allow users to see all Builders, and attach builds to them.
all.Builder = { level = "Read", specific = ["Attach"] }

[[user_group]]
name = "Stack Read"
users = ["user1", "user2"]
# Because of the "Everyone" group, don't need to redefine
# the specific permissions. User will have "Inspect", "Logs", etc.
all.Stack = "Read"

[[user_group]]
name = "Immich Manager"
users = ["user1", "user2"]
# Give per-service management to select users
permissions = [
  { target.type = "Server", target.id = "immich-server", level = "Write", specific = ["Logs", "Inspect", "Terminal"] },
  { target.type = "Stack", target.id = "immich", level = "Write"  }
]

[[user_group]]
name = "Dev Manager"
users = ["user1", "user2"]
# Manage wildcard access to specific resources, in this case with the `dev-` name prefix.
# Note. Doesn't work with Sync "Commit". Only "Execute" direction.
permissions = [
  { target.type = "Server", target.id = "dev-*", level = "Read" },
  { target.type = "Deployment", target.id = "dev-*", level = "Write"  },
  { target.type = "Build", target.id = "dev-*", level = "Write"  },
]

Misc.

• Server: Remove limitations on name. Names can now include Capital letters and spaces. They still have to be unique.
  • Also can use any name for Procedures, Actions, Resource Syncs, Builders and Alerters.
  • Stacks / Deployments / Builds / Repos still have the same naming restrictions (no capitals / spaces)
• Alerter: Ntfy endpoints now support configuring email. Note that you must also make sure SMTP is configured on the Ntfy server. By @FelixBreitweiser in #493
• Resource Sync: Fix issue with User Groups showing "Pending" repeatedly / eroneously.
• UI: Fix the inline rename behavior when renaming multiple resources in a row.
• Startup log: Specify pretty_startup_config = true to get more human readable initial config log.
  • Core Env: KOMODO_PRETTY_STARTUP_CONFIG=true
  • Periphery Env: PERIPHERY_PRETTY_STARTUP_CONFIG=true

Komodo v1.17.5

Changelog

🚨 This release removes support for the ServerTemplate resource. It was an interesting experiment, but ultimately I feel cloud server creation is outside of the scope of Komodo, it wasn't a very refined flow, and is better accomplished using other IAC tools such as OpenTofu.

🚨 Container exec is now enabled even if disable_terminals = true. This allows users to still use container exec while disabling general terminals. If you don't want to allow container exec, you now have to use disable_container_exec in periphery.config.toml (Env: PERIPHERY_DISABLE_CONTAINER_EXEC=true)

🚨 ssl_enabled in periphery.config.toml (Env: PERIPHERY_SSL_ENABLED) now defaults to true. You should update your server address to use https if you haven't yet.

• Websocket: Add disable_websocket_reconnect = true to core.config.toml (Env: KOMODO_DISABLE_WEBSOCKET_RECONNECT=true) if your websocket is causing issues. You can still click the connected indicator in the topbar to manually trigger reconnect. The reconnect logic has also improved to make sure reconnect attempts only happen every 5s, so you might want to try it out before disabling reconnect. Re #368.

• UI: Improve renaming visibility / intuitiveness

  • Now you rename by clicking on the name in the header
  • Lots of people didn't see the previous rename function until it was pointed out, hopefully this makes it easier to figure out.

• Stack: Add the BatchPullStack execution for use in Actions / Procedures.

• Server: Terminals can now be started using your shell of choice (before it was always bash)

• Server: Implemented the execute_terminal api to send command to your Terminals over HTTP call. See below for details.

Execute Terminal

One use case I had in mind for terminals is to have an easy way to handle a simple system update task, say running sudo apt upgrade -y, on a schedule. To do this, I needed a way to interact with terminals from an Action. Now you can do this using komodo.execute_terminal. For example, a simple Action to update all your servers and consolidate / save the logs:

const servers = await komodo.read("ListServers", {
  query: { tags: ["auto-update"] },
});

for (const server of servers) {
  console.log();
  console.log("------------------");
  console.log("Updating", server.name);
  await komodo.write("CreateTerminal", {
    server: server.name,
    name: "apt-upgrade",
    command: "bash",
    recreate: Types.TerminalRecreateMode.DifferentCommand,
  });
  await komodo.execute_terminal(
    {
      server: server.name,
      terminal: "apt-upgrade",
      command:
        "sudo apt update && sudo apt upgrade -y && sudo apt autoremove -y",
    },
    {
      onLine: console.log,
      onFinish: (code) => console.log("Finished:", code),
    },
  );
}

Note. It is necessary to CreateTerminal before you can execute on it, as shown above. Setting recreate: Types.TerminalRecreateMode.DifferentCommand will make it so it does not fail if the terminal already exists, and only recreates the terminal if it was started with a different root command. This allows you to reuse the same terminal and collect the output for later viewing. You can also use Types.TerminalRecreateMode.Always to always run on a fresh terminal.

Other

• The API now supports the Path calling convention, allowing for the request type to be specified in the URL path, which is a bit more concise. For example calling DeployStack:

---------------------------------
## Path calling convention (New)
---------------------------------
curl --header (unchanged) \
    --data '{ "stack": "my-stack", "services": ["server"] }' \ # <- data just matches the rust struct
    https://komodo.example.com/execute/DeployStack # <- request type in path

-----------------------------------
## Type-params calling convention (Old, but still supported)
-----------------------------------
curl --header (unchanged) \
    --data '{ "type": "DeployStack", "params": { "stack": "my-stack", "services": ["server"] } }' \
    https://komodo.example.com/execute

• This is not breaking, the previous way of calling the API continues to be supported.
• The Frontend interface now uses the path based calling convention, as it is much easier to see what calls are happening from the browser devtools during development. That was the main motivation for this feature.
• There isn't any change to the rust or typescript clients.

Komodo v1.17.4

Changelog

Overview

• In app terminal access to your servers re #142
• docker exec into containers re #75
• IMPORTANT: Use disable_terminals (PERIPHERY_DISABLE_TERMINALS) in periphery config to disable this functionality on particular servers.

Details

• Server: Adds the Terminals tab, which allows you to connect to and manage multiple persistent shells on the server.

  • Uses portable-pty for the pseudoterminal on the backend and xterm.js for the frontend.
  • Networked over websockets.
  • Supports TUI applications like htop / ncdu / nvim (and runnables-cli)
  • Each shell history / active running process is persisted on periphery after the client disconnects, making them suitable for long running tasks (you can run servers from them etc)
  • The shell starts as the same linux user that periphery runs as.
    • For systemctl --user installs, you login as your linux user on the host (complete with any custom prompt).
    • For root systemctl installs, you would login as root linux user. You should consider creating a custom periphery user with intented permissions, and updating your periphery.service systemctl config to use this user instead: link
    • For container Periphery, you connect to shell inside periphery container. The functionality will be more limited, but you can still communicate with docker socket in there (its mounted in), and docker exec into containers
  • The terminals can have mutliple Komodo users connected at once, and their view is synced.
  • If Periphery is restarted, the Terminal sessions will be lost, as they are child processes of periphery.
  • User must be admin or have Write permission on Server to connect to terminals
  • Use disable_terminals (PERIPHERY_DISABLE_TERMINALS) in periphery config to disable this functionality on particular servers.
  • Easy access to docker exec -it (container shell access) from Container page, Terminal tab

• Deployment / Stack: Adds the Terminal tab to Deployments and Stack services.

  • Configurable shell command inside container, eg sh or bash.

Komodo v1.17.4-dev-1

🦎 This is pre-release 1 of v1.17.4 for initial testing and feedback. 🦎

It adds the Terminals feature for ssh access. See the ongoing changelog in the PR: #446

Core image: ghcr.io/moghtech/komodo:latest-dev
Periphery image: ghcr.io/moghtech/periphery:latest-dev

Systemd Periphery installer:

• Add --version=v1.17.4-dev-1 to the install command.
• You can remove --version=v1.17.4-dev-1 and run again if you need to go back to the previous version.

curl -sSL \
  https://raw.githubusercontent.com/moghtech/komodo/main/scripts/setup-periphery.py \
  | python3 - --user --version=v1.17.4-dev-1

Komodo v1.17.3

Changelog

• Build: Fixes a bug breaking pre_build functionality, re #438

Thats it for this one, if you don't use this functionality, there is no other change.

Komodo v1.17.2

Changelog

General

• Simplify Periphery directory configuration using one single PERIPHERY_ROOT_DIRECTORY.
  • The change is backward compatible, and your currently configured eg. PERIPHERY_STACK_DIR will still take precedent.
• Resource configuration changes will now show the full TOML diff of the changes made in the Update log

Resources

• Procedure / Action: Implement Schedule feature re #47

  • Supports using English or CRON expression using english-to-cron and croner libraries
  • English Example: Every Sunday at midnight
  • Equivalent CRON: 0 0 0 ? * SUN
  • RunSchedule alert type can be sent when the scheduled runs occur
  • Also add ProcedureFailed and ActionFailed alert types for visibility on failures when running them on schedules.

• Alerter: Add Pushover alerter by @alexshore in #421

Komodo v1.17.2-dev-1

🦎 This is pre-release 1 of v1.17.2 for initial testing and feedback. 🦎

It adds the Schedule feature for Procedures and Actions. See the ongoing changelog in the PR: #409

Core image: ghcr.io/moghtech/komodo:latest-dev
Periphery image: ghcr.io/moghtech/periphery:latest-dev

Systemd Periphery installer:

• Add --version=v1.17.2-dev-1 to the install command.
• You can remove --version=v1.17.2-dev-1 and run again if you need to go back to the previous version.

curl -sSL \
  https://raw.githubusercontent.com/moghtech/komodo/main/scripts/setup-periphery.py \
  | python3 - --user --version=v1.17.2-dev-1