Cannot boot docker image when mount a custom app.ini configuration

[twang2218]

• Gitea version (or commit ref): Since 2eeae84 (master branch)
• Git version: 2.8.5
• Operating system: macOS 10.12.5 + Docker for Mac (17.06)
• Database (use [x]):
  • PostgreSQL
  • MySQL
  • MSSQL
  • SQLite
• Can you reproduce the bug at https://try.gitea.io:
  • Yes (provide example URL)
  • No
  • Not relevant
• Log gist:

Description

Since commit 2eeae84, which belongs to #1471, the docker image, which I built with TAGS="sqlite" make docker, cannot be booted with an app.ini file mounted, and following error will be shown:

$ docker run -it --rm -p 3001:3000 -v $PWD/conf/sqlite/app.ini:/data/gitea/conf/app.ini gitea/gitea:latest
Generating /data/ssh/ssh_host_ed25519_key...
Jul 22 16:58:03 syslogd started: BusyBox v1.24.2
Generating /data/ssh/ssh_host_rsa_key...
Generating /data/ssh/ssh_host_dsa_key...
Generating /data/ssh/ssh_host_ecdsa_key...
Jul 22 16:58:03 sshd[12]: Server listening on :: port 22.
Jul 22 16:58:03 sshd[12]: Server listening on 0.0.0.0 port 22.
2017/07/22 16:58:03 [...s/setting/setting.go:803 NewContext()] [E] Error saving generated JWT Secret to custom config: rename /data/gitea/conf/app.ini.821764490.tmp /data/gitea/conf/app.ini: device or resource busy
2017/07/22 16:58:04 [...s/setting/setting.go:803 NewContext()] [E] Error saving generated JWT Secret to custom config: rename /data/gitea/conf/app.ini.342639761.tmp /data/gitea/conf/app.ini: device or resource busy
...

After dig into the code, I found it's related to this part of code:

https://github.com/go-gitea/gitea/blob/master/modules/setting/setting.go#L831-L838

		cfgSave.Section("security").Key("INTERNAL_TOKEN").SetValue(InternalToken)

		if err := os.MkdirAll(filepath.Dir(CustomConf), os.ModePerm); err != nil {
			log.Fatal(4, "Failed to create '%s': %v", CustomConf, err)
		}
		if err := cfgSave.SaveTo(CustomConf); err != nil {
			log.Fatal(4, "Error saving generated JWT Secret to custom config: %v", err)
		}

cfgSave.SaveTo() is actually create a temporary file first, then write to the temporary file, after that, it then delete the old file, and rename the temporary file to the deleted file name.

I think this kind of operation causes the problem, as we mount the /data/gitea/conf/app.ini file inside the docker, so rm or rename operation will not be allowed. Here is what I tried to manually rm or rename the file:

$ docker run -it --rm -p 3001:3000 -v $PWD/conf/sqlite/app.ini:/data/gitea/conf/app.ini gitea/gitea:latest bash
bash-4.3# cd /data/gitea/conf/
bash-4.3# mount | grep app
osxfs on /data/gitea/conf/app.ini type fuse.osxfs (rw,nosuid,nodev,relatime,user_id=0,group_id=0,allow_other,max_read=1048576)
bash-4.3# rm app.ini
rm: can't remove 'app.ini': Resource busy
bash-4.3# mv app.ini app.ini.tmp
mv: can't rename 'app.ini': Resource busy

The docker image of current version, v1.1.2, can be booted without any problem:

$ docker run -it --rm -p 3001:3000 -v $PWD/conf/sqlite/app.ini:/data/gitea/conf/app.ini gitea/gitea:1.1.2
Jul 22 17:31:02 syslogd started: BusyBox v1.24.2
Generating /data/ssh/ssh_host_ed25519_key...
Generating /data/ssh/ssh_host_rsa_key...
Generating /data/ssh/ssh_host_dsa_key...
Generating /data/ssh/ssh_host_ecdsa_key...
Jul 22 17:31:03 sshd[14]: Server listening on :: port 22.
Jul 22 17:31:03 sshd[14]: Server listening on 0.0.0.0 port 22.
2017/07/22 17:31:03 [T] Custom path: /data/gitea
2017/07/22 17:31:03 [T] Log path: /data/gitea/log
2017/07/22 17:31:03 [I] Gitea v1.1.2 built with: bindata, sqlite
2017/07/22 17:31:03 [I] Log Mode: Console(Trace)
2017/07/22 17:31:03 [I] XORM Log Mode: Console(Trace)
2017/07/22 17:31:03 [I] Cache Service Enabled
2017/07/22 17:31:03 [I] Session Service Enabled
2017/07/22 17:31:03 [I] PING DATABASE sqlite3
2017/07/22 17:31:03 [I] [sql] SELECT name FROM sqlite_master WHERE type='table' and name = ? [args] [version]
...
2017/07/22 17:31:11 Serving [::]:3000 with pid 13
2017/07/22 17:31:11 [I] Listen: http://0.0.0.0:3000

[sapk]

Gitea could not remove the file since it is a mount, I think.
Could you try to mount /data/gitea/conf in place of the direct file ? (like docker run -it --rm -p 3001:3000 -v $PWD/conf/sqlite:/data/gitea/conf gitea/gitea:latest)

@ethantkoenig is allready on it to fix and not create a new file.

[sapk]

For information, this is not specific to that commit. It's just that it need to create a new var to the configuration. If for any reason gitea has to write to the config this will failed even before this commit.

[twang2218]

Yes, it's will be ok if mount on the /data/gitea/conf directory, rather than a file, as the app.ini is not a mount point anymore.

However, it's not the only use case that people just want to mount the single file of app.ini directly to the /data/gitea/conf/app.ini file, there is another case for using gitea, which is inside docker swarm cluster.

When using docker swarm cluster, mounting a local file/directory to a container is not quite reasonable, because the container might be scheduled to another host, which might not have the app.ini locally, so, either use another solution to synchronize the app.ini across all the hosts in the cluster, or use distributed file system, or using docker config for the configuration, which is much better for cluster deployment with rollback capabilities.

Create a config object for app.ini

$ docker config create gitea-conf app.ini

Then create the gitea service with the config file:

$ docker service create
    --name gitea
    -p 80:3000
    --config src=gitea-conf,target="/data/gitea/conf/app.ini"
    gitea/gitea

As you can see, at this point, the configuration object has to be mounted on the file location.

Things get even worse in this case, as the configuration object is read-only. So, even the file will only be updated, rather than be replaced, it will still fail if gitea try to modify the file.

Is that possible we move all the modifications to the database, or other temporary file, and make app.ini immutable?

[sapk]

You are right. This was just to confirm the limitation that we have to take in account. A solution is allready in code review #2255 to fix that