Bug: unable to run Docker container with a volume

[thomas-huegel]

Describe the bug

I am able to run the Docker container in-memory, but not with a Docker volume as persistent storage.

Steps to reproduce

Run the following command, as explained on the documentation page:

docker run --rm --pull always -p 8000:8000 -v /mydata:/mydata surrealdb/surrealdb:latest start file:/mydata/mydatabase.db

Here are the logs I can see:

2024-02-09T13:41:36.643952Z  INFO surreal::env: Running 1.1.1+20240116.b261047 for linux on x86_64
2024-02-09T13:41:36.643985Z  WARN surreal::dbs: ❌🔒 IMPORTANT: Authentication is disabled. This is not recommended for production use. 🔒❌
2024-02-09T13:41:36.644012Z  INFO surrealdb::kvs::ds: Starting kvs store at file:///mydata/mydatabase.db
2024-02-09T13:41:36.645252Z  INFO surrealdb::kvs::ds: Started kvs store at file:///mydata/mydatabase.db
2024-02-09T13:41:36.645267Z ERROR surreal::cli: There was a problem with the database: There was a problem with a datastore transaction: Failed to create RocksDB directory: `Os { code: 13, kind: PermissionDenied, message: "Permission denied" }`.

Expected behaviour

I would expect it to run without error.

SurrealDB version

v1.1.1 for Docker amd64

Contact Details

No response

Is there an existing issue for this?

• I have searched the existing issues

Code of Conduct

• I agree to follow this project's Code of Conduct

[AgustinRamiroDiaz]

Hi!
Are you sure it's not a file permission problem? given that you are using the folder /mydata from your host machine, maybe the process running in the container is not root and cannot access that file due to permissions

This article explains the difference between host and container permissions very well. If you want a quick getaway you can sudo chmod 777 /mydata/ -R in order to grant every user access (in particular you are interested in the user with id 1000) and then it should work.

Another way to bypass this is to run the container with root user

docker run --rm --pull always -p 8000:8000 --user root -v /mydata:/mydata surrealdb/surrealdb:latest start file:/mydata/mydatabase.db

[thomas-huegel]

Hi Agustin.
Thanks for your reply.
I think you emphasize the right problem.
However I have been using dozens of different Docker containers (such as Postgres, Redis, Neo4j etc.) and I had never encountered such a problem before.
chmod 777 is certainly not very secure.
As for --user root, it looks odd to me: is it root on my machine or on the container? or maybe on both? I fear there is some confusion behind the scenes. And why can we run the container as root or not? what kind of choice is this? what does it imply?
Thanks.

[AgustinRamiroDiaz]

@thomas-huegel

As for --user root, it looks odd to me: is it root on my machine or on the container?

--user allows setting the user in the container, so this would mean run the container processes as root (all inside the container)

And why can we run the container as root or not? what kind of choice is this? what does it imply?

As you mentioned the security with the chmod part, to ensure that processes have the correct access inside the container, it is recommended to run them as normal users, so we don't have privileged access to things like rm -rf / in the container.

chmod 777 is certainly not very secure.

I surely agree. I'll share with you other alternative I thought of.

From some tests I did, I found that the default user used in the surrealdb image has id 65532 (probably inherited from the base image cgr.dev/chainguard/glibc-dynamic). You probably have id 1000 in your personal computer.

Also, I've found that when you use docker with volumes (-v) and the folder doesn't exist, it automatically creates it owned by root:

• $ docker run -v $(pwd)/mydata:/mydata surrealdb/surrealdb:latest start file:/mydata/mydatabase.db
  [error logs]
  $ ll
  total 8,0K
  drwxr-xr-x 2 root root 4,0K Feb 10 11:58 mydata/
  

---

Alternatives I though of (I'm omitting some docker parameters for readability):

1. Create the folder prior to the docker command, in order to have it under your user. Then, run the container with your user id so it has the same access to your files

mkdir -p mydata
docker run --user $(id -u) -v $(pwd)/mydata:/mydata surrealdb/surrealdb:latest start file:/mydata/mydatabase.db

2. Send a PR to this repo to set USER 1000 to the built image, in order to set the usual default user for unix systems. This will still cause troubles if your user is not 1000. Also, we should add to the documentation that the folder needs to be created prior to the command execution with proper access

---

I'm still researching if there's a way to create the volume with the container's user permissions, that /would simplify the commands a lot Seems like it's not possible https://stackoverflow.com/a/69507070/13229340. The recommended way is as I mentioned, to prior create the folder

[sgirones]

if you prefer not to do any setup on your end, we provide -dev Docker images that run the SurrealDB process as root and they also provide a shell in case you want to exec into the container.

Other Docker images like the ones you mentioned, change the permissions of the data directory or they just run the process as root.

We can't do that because our production Docker image comes with only the SurrealDB binary. We have no shell to change the permissions of the data directory.

Thanks @AgustinRamiroDiaz for the docs PR 👍

[thomas-huegel]

Thank you for your replies.
I would like to launch it declaratively through Docker Compose and this kind of Yaml file declaring a volume:

version: '3'

services:
  surrealdb:
    container_name: surrealdb-test
    image: docker.io/surrealdb/surrealdb:latest
    command: start --log debug --auth --user root --pass root file:/mydata/database.db
    ports:
      - 8000:8000
    volumes:
      - surrealdb-test:/mydata

volumes:
  surrealdb-test:

But I am currently unable to do it, neither with rootless Docker nor with rootful Docker.
It does work however with podman-compose…
Any clue, please?

Remark: that you do not follow common practices might be very appalling to newcomers, whatever your rationale.

[AgustinRamiroDiaz]

@thomas-huegel I think that you ar putting the user flag in the wrong place

Docker compose has a key you can set, so it'll be something like

services:
  surrealdb:
    user: root

And then take out the --user root from the command

[thomas-huegel]

Ok, thanks, and what about pass then?

services.surrealdb Additional property pass is not allowed

[AgustinRamiroDiaz]

@thomas-huegel sorry, there was a confusion here. There's the user configuration from docker, which was the one we talked about a lot in this thread, and then there's the user and pass configurations for surrealdb cli

I was proposing that you add the docker user configuration. Here's the change to your docker-compose (I changed the names of the user and password of surrealdb so there's no confusion)

version: '3'

services:
  surrealdb:
    user: root
    container_name: surrealdb-test
    image: docker.io/surrealdb/surrealdb:latest
    command: start --log debug --auth --user adminuser --pass adminpassword file:/mydata/database.db
    ports:
      - 8000:8000
    volumes:
      - surrealdb-test:/mydata

volumes:
  surrealdb-test:

[thomas-huegel]

So, you want surrealdb to become root of my machine (with rootful Docker), or at least of my account (with rootless Docker)?
I am not sure I should accept this…

[AgustinRamiroDiaz]

So, you want surrealdb to become root of my machine (with rootful Docker), or at least of my account (with rootless Docker)? I am not sure I should accept this…

no no, I just want to make the container process owner root inside the container, nothing to do with your host machine

Maybe the official docker reference is clearer than me, you can check it out at https://docs.docker.com/engine/reference/commandline/container_run/#options

[thomas-huegel]

Quoting https://docs.docker.com/engine/reference/run/

The default user within a container is root (uid = 0). You can set a default user to run the first process with the Dockerfile USER instruction. When starting a container, you can override the USER instruction by passing the -u option.

[AgustinRamiroDiaz]

Quoting https://docs.docker.com/engine/reference/run/

The default user within a container is root (uid = 0). You can set a default user to run the first process with the Dockerfile USER instruction. When starting a container, you can override the USER instruction by passing the -u option.

Exactly, that's what I mean. The thing is that the default user is overwritten as explained here, so you need to manually overwrite it if you want it to be able to read your /mydata folder (you could also use your user instead of root, taking account the caveats explained in the comment)

[thomas-huegel]

This is also what I mean and what I fear.
See for example https://docs.docker.com/engine/security/#docker-daemon-attack-surface for an overview of what root can do.

[AgustinRamiroDiaz]

This is also what I mean and what I fear. See for example https://docs.docker.com/engine/security/#docker-daemon-attack-surface for an overview of what root can do.

I understand your concern now. Then you can follow the approach of setting you user and group ID and managing the permissions of /mydata to be owned by your user

[thomas-huegel]

But is it possible to do that declaratively by Docker Compose?
Isn't the Dockerfile the right place to do that?

[AgustinRamiroDiaz]

But is it possible to do that declaratively by Docker Compose?

yes, you can do so with the user key
check out https://stackoverflow.com/questions/56844746/how-to-set-uid-and-gid-in-docker-compose for the syntax

version: '3'

services:
  surrealdb:
    user: <thomas huegel user id and group id here>
    container_name: surrealdb-test
    image: docker.io/surrealdb/surrealdb:latest
    command: start --log debug --auth --user adminuser --pass adminpassword file:/mydata/database.db
    ports:
      - 8000:8000
    volumes:
      - surrealdb-test:/mydata

volumes:
  surrealdb-test:

[thomas-huegel]

I get the same error with user: "1000" or user: "1000:1000":

surrealdb_test  | 2024-02-20T06:25:25.648482Z ERROR surreal::cli: There was a problem with the database: There was a problem with a datastore transaction: Failed to create RocksDB directory: `Os { code: 13, kind: PermissionDenied, message: "Permission denied" }`.