Skip to content

Bug: OpenCode fails to start when userspace is backed by JuiceFS (multi-node) #3390

Description

@bayerhazard

Environment

  • Olares 1.12.6, multi-node (control-plane + 1 worker)
  • Userspace provisioned with olares-cli install --enable-juicefs
    (JuiceFS FUSE mounted at /olares/rootfs, data backend: local managed-MinIO)
  • App: opencode 1.0.18 (this chart), image beclab/opencode:1.18.9.1

Symptom

Pod stays in Init:CrashLoopBackOff. Init container init-overlay
(privileged, runs /opt/olares-bake/lib/init-overlay.sh) exits 255 with:

mount: mounting overlay on /pkg-overlay/merged/usr failed: Not supported

Root cause

  • In multi-node mode the userspace (/olares/rootfs/...) is served by
    JuiceFS = a FUSE filesystem (fuse.juicefs).
  • This chart places all overlay-dependent paths under the userspace:
    • pkg-overlay upper/work dirs -> {{ .Values.userspace.appData }}/.pkg-overlay
    • dind-storage (dockerd /var/lib/docker) -> {{ .Values.userspace.appData }}
    • the pkg-* merged views -> .../.pkg-overlay/merged/*
      (see templates/opencode.yaml, init-overlay ~L74-92 and volumes ~L395-470)
  • OverlayFS cannot use a FUSE filesystem as its upperdir/workdir ->
    kernel returns EOPNOTSUPP ("Not supported"). The same applies to
    docker-dind, which uses the default overlay2 storage driver on the
    JuiceFS-backed docker dir and will fail once init-overlay is bypassed.
  • values.yaml exposes no knob (only replicaCount), so the paths cannot
    be relocated at install time.
  • Confirmed working when the userspace is a plain hostPath on a real block
    device (single-node / hostpath-userspace). This is a FUSE limitation,
    not an opencode-specific bug.

Impact

Any pkga-based app that uses init-overlay (or dockerd with overlay2) on a
JuiceFS userspace cannot install/start. Multi-node is the default Olares
architecture for multiple nodes, so this blocks opencode there entirely.

Suggested fixes (any of)

  1. Preferred: relocate the overlay-dependent volumes (pkg-overlay,
    dind-storage) to a node-local real-disk hostPath (e.g. under the local
    appCache/userData area) instead of the JuiceFS userspace. The overlay
    upper layer is inherently node-local, so this stays compatible with the
    shared JuiceFS userspace for actual user data.
  2. Add a values/OlaresManifest knob that selects where pkg-overlay and
    docker storage live.
  3. Make init-overlay.sh detect an unsupported (FUSE) upperdir and fall back
    to bind mounts (degraded: no persistent pkg overlay).

Repro

  1. Multi-node Olares 1.12.6 installed with --enable-juicefs.
  2. Install opencode 1.0.18 from the store.
  3. Watch init-overlay CrashLoopBackOff with the mount error above.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions