Skip to content

Document app data storage locations and migration procedures#7

Merged
mairas merged 1 commit into
mainfrom
docs/app-data-and-migration
Feb 22, 2026
Merged

Document app data storage locations and migration procedures#7
mairas merged 1 commit into
mainfrom
docs/app-data-and-migration

Conversation

@mairas

@mairas mairas commented Feb 22, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Adds a new App Data and Migration page to the User Guide explaining where container apps store persistent data (/var/lib/container-apps/<package-name>/data/), the general migration procedure, and backup/restore instructions
  • Adds a Migrating from an existing system section to the Marine Apps page with step-by-step walkthroughs for Signal K, InfluxDB (2.x and 1.x), Grafana, and AvNav
  • The two pages cross-link: the generic page points to the marine migration section for app-specific details, and vice versa

Test plan

  • Verify pages render correctly on the deployed docs site
  • Check cross-links between app-data.md and marine-apps.md work
  • Verify the anchor link to marine-apps.md#migrating-from-an-existing-system resolves correctly

🤖 Generated with Claude Code

@mairas

mairas commented Feb 22, 2026

Copy link
Copy Markdown
Contributor Author

Adversarial Review

Gut Reaction

Solid documentation PR that covers a genuine user need. The data paths and package names are correct, but several of the step-by-step migration commands have issues that would cause failures or data problems for users following them literally.

Critical Issues

1. scp commands will fail with permission denied

  • Location: docs/user-guide/marine-apps.md lines 125-126, 158-159, 162-163, 194-195, 217-218; docs/user-guide/app-data.md line 65
  • Problem: All scp commands write directly to paths under /var/lib/container-apps/ which is root-owned. Running scp -r user@old-system:/path/* /var/lib/container-apps/... as a regular user will fail with "Permission denied". Even with sudo earlier in the sequence, scp itself runs as the invoking user.
  • Risk: Every user following these migration guides will hit this error on the very first data-copy step.
  • Suggestion: Either (a) instruct users to scp to a temp directory first, then sudo cp, or (b) use ssh user@old-system 'tar czf - /path/to/data' | sudo tar xzf - -C /target/, or (c) note at the top that all commands should be run as root (sudo -i or sudo su -). Option (c) is simplest since the systemctl and chown commands already require root.

2. "If there is no user: field, the container runs as root" is incorrect

  • Location: docs/user-guide/app-data.md line 70
  • Problem: This claim is wrong. Many Docker images specify a non-root user in their Dockerfile (via USER directive), not in docker-compose.yml. Signal K is a concrete example in this very PR: its docker-compose.yml has NO user: field, yet it runs as UID 1000 (the node user, set in the upstream Dockerfile). The prestart.sh confirms this with chown -R 1000:1000.
  • Risk: A user applying the stated rule to Signal K would conclude it runs as root and set ownership to root:root, breaking the container.
  • Suggestion: Either remove this rule entirely, or reword to something like: "Check the docker-compose.yml user: field first. If absent, the image's Dockerfile determines the runtime user -- consult the prestart.sh or the upstream image documentation."

3. Signal K migration may break reverse proxy and OIDC setup

  • Location: docs/user-guide/marine-apps.md lines 120-134
  • Problem: Blindly copying settings.json from an old system will overwrite the HaLOS-specific settings that the prestart script creates on first boot (ssl: false, trustProxy: true, the gpsd provider configuration). The prestart.sh only creates settings.json if the file does not exist, so the migrated copy will persist and Signal K will likely fail behind Traefik (it will try to handle TLS itself, or reject proxied requests).
  • Risk: Signal K becomes inaccessible after migration because it doesn't trust the reverse proxy.
  • Suggestion: Add a warning that users should either (a) merge their old settings.json with the HaLOS defaults, or (b) copy everything except settings.json and reconfigure connections manually. Specifically, "ssl": false and "trustProxy": true must be present in the HaLOS copy.

Major Concerns

4. InfluxDB 1.x migration advice is oversimplified and potentially misleading

  • Location: docs/user-guide/marine-apps.md lines 174-181
  • Problem: The influx_inspect export + influx write path only migrates raw data. It does not preserve retention policies, continuous queries, users, or database permissions. The linked InfluxDB docs describe a much more involved influxd upgrade process. Presenting this as a simple two-step procedure may lead to incomplete migrations.
  • Suggestion: Either expand the section to mention what IS and IS NOT migrated (retention policies, CQs, user accounts are lost), or simply defer to the linked docs without summarizing them inaccurately.

5. Grafana migration has unmentioned OIDC/authentication conflict

  • Location: docs/user-guide/marine-apps.md lines 183-206
  • Problem: After copying grafana.db from a bare-metal Grafana, the database will contain old user accounts and password hashes. HaLOS Grafana is configured for Authelia OIDC SSO. The migrated grafana.db may have conflicting authentication settings (e.g., GF_AUTH_GENERIC_OAUTH settings stored in the DB overriding environment variables, or old admin accounts interfering with OIDC).
  • Suggestion: Add a note that after migration, users should verify OIDC login still works and may need to update or remove old local Grafana accounts.

Minor Issues

6. Traefik data path table is incomplete

  • Location: docs/user-guide/app-data.md line 45
  • Problem: The table only lists .../data/traefik/certs/ for TLS certificates, but Traefik also stores acme.json at .../data/traefik/acme.json (for Let's Encrypt / ACME certificates). This is user-relevant backup data.
  • Suggestion: Add acme.json to the Traefik row, or add a second row.

7. Authelia Valkey data not mentioned

  • Location: docs/user-guide/app-data.md lines 43-44
  • Problem: Authelia's row mentions users_database.yml, OIDC config, session data, but there's also a Valkey (Redis) data directory at .../data/authelia/valkey/ that stores session persistence data. Probably not critical for backup, but worth noting for completeness.
  • Suggestion: Add "Valkey session store" to the contents column, or ignore if you consider it ephemeral.

8. InfluxDB chown -R root:root step is unnecessary noise

  • Location: docs/user-guide/marine-apps.md lines 165-168
  • Problem: Since InfluxDB runs as root in the container (no user: field, no Dockerfile USER override for influxdb:2.8.0), and the directories are already root-owned after being created by the postinst script, the chown step does nothing useful. It adds a step that users might wonder about.
  • Suggestion: Either remove the chown step for InfluxDB, or change the comment to explain why it's there (e.g., "ensure ownership is correct after scp which may have changed it").

9. Backup tar restore is potentially destructive

  • Location: docs/user-guide/app-data.md lines 92-101
  • Problem: sudo tar xzf backup.tar.gz -C / restores with absolute paths. If the backup was created with the shown command (tar czf ... /var/lib/container-apps/), tar stores absolute paths. This is fine, but there's no mention that this will overwrite ALL existing data without confirmation. And there's no advice to back up the current state first.
  • Suggestion: Add a brief note that this overwrites existing data, and suggest backing up the current state before restoring.

Questions

Q1: Is casaos-uptimekuma-container (app-data.md line 24) the actual package name that would be generated for a CasaOS Uptime Kuma app? The container-packaging-tools CasaOS converter would use the CasaOS app ID as the app_id with a casaos prefix. I am not sure if the actual generated name would use "uptimekuma" (no hyphen) or "uptime-kuma".

Q2: For Signal K, the doc lists node_modules/ as something to migrate. Is this actually advisable? Signal K plugin versions may be incompatible with the container's Node.js version. Would it be better to recommend copying only configuration and letting Signal K reinstall plugins?

Q3: The AvNav source path /home/pi/avnav/data/* assumes a bare-metal Pi install. Is this the most common migration scenario? Users migrating from an AvNav Docker setup would have a different source path.

What's Actually Good

  • Data path table is correct: Cross-referenced every path against the actual docker-compose.yml files. Package names, volume mount paths, and container paths all match the source code.
  • Systemd service names are correct: The systemctl stop/start <package-name> commands use the correct service names (confirmed against the systemd service template which uses {{ package.name }}.service).
  • CONTAINER_DATA_ROOT expansion is correctly documented: The docs correctly state the pattern /var/lib/container-apps/<package-name>/data/ which matches the builder.py line CONTAINER_DATA_ROOT="/var/lib/container-apps/{package_name}/data".
  • Cross-links between pages: The bidirectional links between app-data.md and marine-apps.md are well structured. The MkDocs anchor #migrating-from-an-existing-system will resolve correctly.
  • Core containers base path: Correctly identified as /var/lib/container-apps/halos-core-containers/ (not using the marine naming pattern).
  • Grafana UID 472: Correct per the docker-compose.yml user: "472" and confirmed by the prestart.sh chown -R 472:472.
  • The apt remove/purge distinction (app-data.md line 13) is a good detail that users need to know.
  • Single atomic commit: Clean commit history with one logical change.

New "App Data and Migration" page in the User Guide explains where
container apps store persistent data and how to back up and restore it.
Marine-specific migration walkthroughs (Signal K, InfluxDB, Grafana,
AvNav) are added to the Marine Apps page.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@mairas mairas force-pushed the docs/app-data-and-migration branch from e1acd5e to b2461f3 Compare February 22, 2026 21:18
@mairas mairas merged commit a4722f7 into main Feb 22, 2026
@mairas mairas deleted the docs/app-data-and-migration branch May 27, 2026 14:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant