chttpd_auth_cache changes listener died database_does_not_exist at mem3_shards

[clewlb]

I keep getting these from the docker console when running off the mint couchdb image:

The command:
docker run -p 5984:5984 couchdb

The console log:

[notice] 2018-01-05T03:02:44.977427Z nonode@nohost <0.326.0> -------- chttpd_auth_cache changes listener died database_does_not_exist at mem3_shards:load_shards_from_db/6(line:403) <= mem3_shards:load_shards_from_disk/1(line:378) <= mem3_shards:load_shards_from_disk/2(line:407) <= mem3_shards:for_docid/3(line:91) <= fabric_doc_open:go/3(line:38) <= chttpd_auth_cache:ensure_auth_ddoc_exists/2(line:187) <= chttpd_auth_cache:listen_for_changes/1(line:134)
[error] 2018-01-05T03:02:49.978270Z nonode@nohost emulator -------- Error in process <0.3612.0> with exit value: {database_does_not_exist,[{mem3_shards,load_shards_from_db,"_users",[{file,"src/mem3_shards.erl"},{line,403}]},{mem3_shards,load_shards_from_disk,1,[{file,"src/mem3_shards.erl"},{line,378}]},{mem3_shards,load_shards_from_disk...

These errors keep repeating themselves.

My docker info:
Containers: 2
Running: 0
Paused: 0
Stopped: 2
Images: 1
Server Version: 17.09.1-ce
Storage Driver: overlay2
Backing Filesystem: extfs
Supports d_type: true
Native Overlay Diff: true
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
Volume: local
Network: bridge host ipvlan macvlan null overlay
Log: awslogs fluentd gcplogs gelf journald json-file logentries splunk syslog
Swarm: inactive
Runtimes: runc
Default Runtime: runc
Init Binary: docker-init
containerd version: 06b9cb35161009dcb7123345749fef02f7cea8e0
runc version: 3f2f8b84a77f73d38244dd690525642a72156c64
init version: 949e6fa
Security Options:
seccomp
Profile: default
Kernel Version: 4.9.49-moby
Operating System: Alpine Linux v3.5
OSType: linux
Architecture: x86_64
CPUs: 4
Total Memory: 1.952GiB
Name: moby
ID: BDAD:JKIS:2H2B:YHQK:HNHE:EPJD:NGVJ:UJCI:44SL:GRKT:WJQZ:AA4O
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): true
File Descriptors: 19
Goroutines: 31
System Time: 2018-01-05T03:20:54.59263177Z
EventsListeners: 1
No Proxy: *.local, 169.254/16
Registry: https://index.docker.io/v1/
Experimental: true
Insecure Registries:
127.0.0.0/8
Live Restore Enabled: false

My hardware is:
Model Name: MacBook Pro
Model Identifier: MacBookPro11,2
Processor Name: Intel Core i7
Processor Speed: 2.3 GHz
Number of Processors: 1
Total Number of Cores: 4
L2 Cache (per Core): 256 KB
L3 Cache: 6 MB
Memory: 16 GB
Boot ROM Version: MBP112.0142.B00
SMC Version (system): 2.18f15
Serial Number (system): C02MG1NYFD58
Hardware UUID: A111CE82-C171-525C-884E-137091466666

OS is:
System Version: macOS 10.13.2 (17C88)
Kernel Version: Darwin 17.3.0
Boot Volume: Macintosh HD
Boot Mode: Normal

[clewlb]

Adding _user db solves the issue.

[8ivek]

Clewlb,
I am getting this same error,
where should we add _user db ?
Thanks

[wohali]

Just do PUT /_users with your favourite client. The same is needed for _replicator.

All of this is documented.

[clewlb]

@8ivek, just add a _user database into your couchdb instance(node). Like this:

curl -X PUT http://127.0.0.1:5984/_users

[8ivek]

Thanks guys :)

[hanyu100]

Thanks , It's worked

[hanyu100]

[iugo]

same issue:

[error] 2018-10-26T08:08:18.801555Z couchdb@0.0.0.0 emulator -------- Error in process <0.793.0> on node 'couchdb@0.0.0.0' with exit value:
couchdbbb_1 | {database_does_not_exist,[{mem3_shards,load_shards_from_db,"_users",[{file,"src/mem3_shards.erl"},{line,395}]},{mem3_shards,load_shards_from_disk,1,[{file,"src/mem3_shards.erl"},{line,370}]},{mem3_shards,load_shards_from_disk,2,[{file,"src/mem3_shards.erl"},{line,399}]},{mem3_shards,for_docid,3,[{file,"src/mem3_shards.erl"},{line,86}]},{fabric_doc_open,go,3,[{file,"src/fabric_doc_open.erl"},{line,38}]},{chttpd_auth_cache,ensure_auth_ddoc_exists,2,[{file,"src/chttpd_auth_cache.erl"},{line,187}]},{chttpd_auth_cache,listen_for_changes,1,[{file,"src/chttpd_auth_cache.erl"},{line,134}]}]}

[j13k]

Just do PUT /_users with your favourite client. The same is needed for _replicator.

All of this is documented.

Setup documentation URL has changed:
http://docs.couchdb.org/en/latest/setup/index.html

[happilymarrieddad]

Hey everybody,

Is there a reason this isn't done in the dockerfile? It seems like it's something that should be part of the startup script especially if everyone using the container image has to do it before they can use the image. Just curious. Thanks!

[wohali]

See #142. This is an issue for upstream CouchDB, not the Docker image.

[ThibaultJanBeyer]

Is there a way to pre-create these missing databases for example using the .ini file? I'm having hard times to create databases from within the docker image :(
+1 for #142
@happilymarrieddad would you mind sharing your script that does that if you have one?

Edit:

Here is how I solved it:

Docker File

FROM couchdb:2

COPY ./prod.ini /opt/couchdb/etc/local.d/
COPY ./prod.sh /usr/local/bin/

EXPOSE 5984

CMD /usr/local/bin/prod.sh & /opt/couchdb/bin/couchdb

prod.sh

#!/usr/bin/env bash
sleep 10

curl -X PUT http://localhost:5984/_users
curl -X PUT http://localhost:5984/_replicator
curl -X PUT http://localhost:5984/_global_changes

[wohali]

When a solution for apache/couchdb#1977 lands, we'll support it in couchdb-docker.

[suculent]

@ThibaultJanBeyer just a question (i'm trying your solution)... how can this work, if you're doing PUT before starting the couchdb?

> CMD /usr/local/bin/prod.sh & /opt/couchdb/bin/couchdb
>
> curl -X PUT http://localhost:5984/_users
> curl -X PUT http://localhost:5984/_replicator
> curl -X PUT http://localhost:5984/_global_changes

[ThibaultJanBeyer]

because there is a 10 seconds sleep

#!/usr/bin/env bash
sleep 10
…

[happilymarrieddad]

@ThibaultJanBeyer hey, I just started having the services using it calling a script if the tables are not there. I already needed to add some other tables so I make a query against the database and if it returns no tables then I call the generation script. Probably not the best way but I've had it in production for several months and have had no issues with Kubernetes auto-scaling and what not.

[scottohara]

When a solution for apache/couchdb#1977 lands, we'll support it in couchdb-docker.

It would appear that this change landed in 3.0 ([couchdb] single_node=true).

Does that mean we should expect something like a COUCHDB_SINGLE_NODE environment variable to be supported at some point for couchdb-docker?

Or will the image default to [couchdb] single_node=true?

[wohali]

No, A very large number of people use the image for clusters.

Just mount a local.ini or local.d/10-single-node.ini file into your image with the necessary new setting.

[jkhsjdhjs]

I'd really love to see a COUCHDB_SINGLE_NODE environment variable like @scottohara proposed, so I can run couchdb-docker with gitlab CI without errors. Currently the couchdb setup eventually times out, which makes my test run a bit longer than necessary.
Unfortunately it doesn't seem to be possible to mount files inside the image or execute a script before the actual docker entrypoint is executed, so unless I'm creating my own image, there seems to be no way to run a single node couchdb instance within gitlab CI (without errors).
Please prove me wrong if I am, but I just haven't found any other solution to this.
Also I'd like to hear your opinion on an environment variable. It would be a great addition in my opinion, since it would extend the entrypoint by just a few lines.

[wohali]

@jkhsjdhjs Why doesn't this work?

echo "[couchdb]" >10-single-node.ini
echo "single_node=true" >> 10-single-node.ini
docker run -p 5984:5984 -v /path/to/10-single-node.ini:/opt/couchdb/etc/local.d/10-single-node.ini apache/couchdb:3.1.0

Now everything is there:

$ curl http://admin:password@localhost:5984/_all_dbs
["_replicator","_users"]

[jkhsjdhjs]

Thanks for your quick reply! With your solution I'd have to switch to an image where the docker command is available. I was using the python:3.6 image and switched to gitlab/dind to test your solution.
Docker setup doesn't really seem to work:

2020-06-14T21:01:49.968285094Z mount: permission denied (are you root?)
2020-06-14T21:01:49.968306194Z Could not mount /sys/kernel/security.
2020-06-14T21:01:49.968317594Z AppArmor detection and --privileged mode might break.
2020-06-14T21:01:49.969062288Z mount: permission denied (are you root?)

Afterwards it downloads the docker image just fine, but is unable to start it:

[os_mon] memory supervisor port (memsup): Erlang has closed
[os_mon] cpu supervisor port (cpu_sup): Erlang has closed
{"Kernel pid terminated",application_controller,"{application_start_failure,config,{{shutdown,{failed_to_start_child,config,{{case_clause,{error,eisdir}},[{config,parse_ini_file,1,[{file,\"src/config.erl\"},{line,336}]},{config,'-init/1-fun-0-',1,[{file,\"src/config.erl\"},{line,228}]},{lists,map,2,[{file,\"lists.erl\"},{line,1239}]},{lists,map,2,[{file,\"lists.erl\"},{line,1239}]},{config,init,1,[{file,\"src/config.erl\"},{line,227}]},{gen_server,init_it,6,[{file,\"gen_server.erl\"},{line,328}]},{proc_lib,init_p_do_apply,3,[{file,\"proc_lib.erl\"},{line,247}]}]}}},{config_app,start,[normal,[]]}}}"}
Kernel pid terminated (application_controller) ({application_start_failure,config,{{shutdown,{failed_to_start_child,config,{{case_clause,{error,eisdir}},[{config,parse_ini_file,1,[{file,"src/config.er
Crash dump is being written to: erl_crash.dump...done
/dev/mapper/control: open failed: Operation not permitted
00:02
Failure to communicate with kernel device-mapper driver.
Check that device-mapper is available in the kernel.
Command failed
mount: permission denied
Could not mount /sys/kernel/security.
AppArmor detection and --privileged mode might break.
mkdir: cannot create directory '/sys/fs/cgroup/name=systemd': Read-only file system
mount: mount point /sys/fs/cgroup/name=systemd is not a directory
ln: failed to create symbolic link '/sys/fs/cgroup/systemd/name=systemd': Read-only file system

[wohali]

You should open a new Discussion on this here:

https://github.com/apache/couchdb/discussions

as your issue is unrelated to the request filed on this topic, nor is it related to "single node mode."

[jkhsjdhjs]

For reference, a link to the discussion I just started: apache/couchdb#2948