Insufficient free space for journal files

[jareware]

I'm having trouble starting the image on Ubuntu:

$ uname -r
3.13.0-30-generic
$ cat /etc/issue
Ubuntu 14.04 LTS \n \l

...though it runs fine in a boot2docker box for example.

$ docker run -d dockerfile/mongodb
76814a1501cc77df70283041143479a749a7ed9ffa46558e7dad00302374cdb8
$ docker logs 76814a1501cc77df70283041143479a749a7ed9ffa46558e7dad00302374cdb8
mongod --help for help and startup options
2014-07-15T12:59:41.035+0000 [initandlisten] MongoDB starting : pid=1 port=27017 dbpath=/data/db 64-bit host=76814a1501cc
2014-07-15T12:59:41.035+0000 [initandlisten] db version v2.6.1
2014-07-15T12:59:41.035+0000 [initandlisten] git version: 4b95b086d2374bdcfcdf2249272fb552c9c726e8
2014-07-15T12:59:41.035+0000 [initandlisten] build info: Linux build14.nj1.10gen.cc 2.6.32-431.3.1.el6.x86_64 #1 SMP Fri Jan 3 21:39:27 UTC 2014 x86_64 BOOST_LIB_VERSION=1_49
2014-07-15T12:59:41.035+0000 [initandlisten] allocator: tcmalloc
2014-07-15T12:59:41.036+0000 [initandlisten] options: {}
2014-07-15T12:59:41.089+0000 [initandlisten] journal dir=/data/db/journal
2014-07-15T12:59:41.089+0000 [initandlisten] recover : no journal files present, no recovery needed
2014-07-15T12:59:41.089+0000 [initandlisten]
2014-07-15T12:59:41.089+0000 [initandlisten] ERROR: Insufficient free space for journal files
2014-07-15T12:59:41.089+0000 [initandlisten] Please make at least 3379MB available in /data/db/journal or use --smallfiles
2014-07-15T12:59:41.090+0000 [initandlisten]
2014-07-15T12:59:41.090+0000 [initandlisten] exception in initAndListen: 15926 Insufficient free space for journals, terminating
2014-07-15T12:59:41.090+0000 [initandlisten] dbexit:
2014-07-15T12:59:41.090+0000 [initandlisten] shutdown: going to close listening sockets...
2014-07-15T12:59:41.090+0000 [initandlisten] shutdown: going to flush diaglog...
2014-07-15T12:59:41.091+0000 [initandlisten] shutdown: going to close sockets...
2014-07-15T12:59:41.091+0000 [initandlisten] shutdown: waiting for fs preallocator...
2014-07-15T12:59:41.091+0000 [initandlisten] shutdown: lock for final commit...
2014-07-15T12:59:41.091+0000 [initandlisten] shutdown: final commit...
2014-07-15T12:59:41.091+0000 [initandlisten] shutdown: closing all files...
2014-07-15T12:59:41.091+0000 [initandlisten] closeAllFiles() finished
2014-07-15T12:59:41.091+0000 [initandlisten] journalCleanup...
2014-07-15T12:59:41.092+0000 [initandlisten] removeJournalFiles
2014-07-15T12:59:41.122+0000 [initandlisten] shutdown: removing fs lock...
2014-07-15T12:59:41.122+0000 [initandlisten] dbexit: really exiting now

Any ideas? I was under the impression the default container virtual size limit was around 10 GB, though I'm having a hard time finding a source for that...

[jareware]

Changing the CMD to CMD ["mongod", "--smallfiles"] fixes the issue, but would be nice to understand what's causing the inconsistency depending on the host.

[boscoMW]

@jareware instead of modifying the docker file you can run the container this way

docker run -d -p 27017:27017 --name test-mongo mongo mongod --smallfiles

[davclark]

I still don't understand why you need smallfiles. So, I'd vote for re-opening the issue.

[lzilioli]

@davclark +1

[davclark]

I'm fairly certain my issue was caused by using boot2docker, which is surprising given the OP.

Basically, -v is mapping to a directory in the boot2docker-vm, which does not in fact have much space available.

A message that would have saved me some confusion would have been to print out a slightly more informative message with the approximate free space available.

[phuongnd08]

I increased my boot2docker hard drive to 60GB, and have no problem running this command dd if=/dev/zero of=sample_big_file.txt bs=1 count=1 se ek=5000000000 (creating 5G) on the boot2docker machine. But if I run a bash inside the mongo, then:

Wed Oct 22 10:56:18.229 [initandlisten] journal dir=/data/db/journal
Wed Oct 22 10:56:18.229 [initandlisten] recover : no journal files present, no recovery needed
Wed Oct 22 10:56:18.230 [initandlisten]
Wed Oct 22 10:56:18.230 [initandlisten] ERROR: Insufficient free space for journal files
Wed Oct 22 10:56:18.230 [initandlisten] Please make at least 3379MB available in /data/db/journal or use --smallfiles

$ df -h
rootfs           57G   20G   35G  37% /
none             57G   20G   35G  37% /
tmpfs          1005M     0 1005M   0% /dev
shm              64M     0   64M   0% /dev/shm
/dev/sda1        57G   20G   35G  37% /etc/resolv.conf
/dev/sda1        57G   20G   35G  37% /etc/hostname
/dev/sda1        57G   20G   35G  37% /etc/hosts
tmpfs           1.8G  240M  1.6G  14% /data/db
tmpfs          1005M     0 1005M   0% /proc/kcore

$ ls -l /data/db/journal/
total 4
-rw-r--r-- 1 mongodb root 5000000001 Oct 22 10:55 sample_big_file.txt

Apparently docker is imposing some incorrect limit on the /data/db folder?

[Globegitter]

I am getting the same error - it worked for me multiple times to run the container and now all off the sudden it decided to throw me this error without any apparent reason.

[Globegitter]

Is anyone finding a proper solution for this, that does not require smallfiles?

[giladbeeri]

Happens also to me, with OS X and boot2docker 1.3.1. and --smallfiles doesn't solve the issue.

Command-line is:
docker run -d --name mongo-server -p 27017:27017 -v /data/mongodb/db:/data/db mongo-image mongod --smallfiles --replSet rs0

And I get the error:

2014-11-06T09:34:38.480+0000 [initandlisten] ERROR: Insufficient free space for journal files
2014-11-06T09:34:38.480+0000 [initandlisten] Please make at least 3379MB available in /data/db/journal or use --smallfiles
2014-11-06T09:34:38.480+0000 [initandlisten]
2014-11-06T09:34:38.483+0000 [initandlisten] exception in initAndListen: 15926 Insufficient free space for journals, terminating

And:

docker@boot2docker:~$ df -h
Filesystem                Size      Used Available Use% Mounted on
rootfs                    1.8G    204.6M      1.6G  11% /
tmpfs                     1.8G    204.6M      1.6G  11% /
tmpfs                  1004.2M         0   1004.2M   0% /dev/shm
/dev/sda1                32.6G      4.2G     26.7G  14% /mnt/sda1
cgroup                 1004.2M         0   1004.2M   0% /sys/fs/cgroup
none                    233.0G    126.6G    106.3G  54% /Users
/dev/sda1                32.6G      4.2G     26.7G  14% /mnt/sda1/var/lib/docker/devicemapper```

UPDATE:
If I mount the directory to /data/mongo/db, it errs, because it's only 1.8GB in size. If I mount it to /mnt/sda1/data/mongo/db, it works. This is a mount point that was created automatically because I added to boot2docker's config the line DiskSize = 35000.

So the solution for me was to set /mnt/sda1 as the volume for my docker volumes.

[bicccio]

this is my fig yml right now:

db:
image: mongo:latest
ports:

• "27017:27017"

what should be the content of command??
db:
image: mongo:latest
ports:

• "27017:27017"
  commands: --smallfiles

Is this correct?

[chacken]

@bicccio remove the s in commands

command: --smallfiles

P.S. this is an ongoing issue that should probably get a fix. Shouldn't have to be using --smallfiles

[eloone]

I mounted a directory from the host ex. /host/data to the container to solve the issue:
docker run -v /host/data:/data/db mongo

[aboutvale]

I have the same problem in AWS EC2 with Amazon Linux. Even though I mounted a directory from the host as suggested by @eloone it doesn't work.
/data/db in my host is a mountpoint where I mounted a 50GB EBS disk.
Then I run docker run --name mongo -d -v /data/db:/data/db mongo but the error is the same, moreover if I run
docker run --name some-mongo -d -v /data/db:/data/db mongo df -h
I get
/dev/xvda1 7.8G 6.0G 1.8G 78% /data/db
The same behaviour if I use a data volume container. Actually I don't want to use --smallfiles, what is the right way to run a dockerized mongo in a production environment with data volume container avoiding space problems?

[JC1738]

Have the same problem with aws ec2, any update on this?

[jareware]

I'm using the official image (https://registry.hub.docker.com/_/mongo/) now, no issues, very much recommended.

[h12w]

Destroy and recreate the boot2docker VM and pull mongo:latest again, the issue disappears for now.

boot2docker destroy
boot2docker init

[aboutvale]

Ok, it seems ok. Thank you @jareware and @HaiLiang.

[michael-erasmus]

Still now fix for this? I have to resort to destroying and recreating my VM like @HaiLiang mentions, but after a while it happens again.

--smallfiles didn't work either

[jareware]

@michael-erasmus, have you tried the official image?

[ksylvan]

I'm getting this on my Linux Fedora installation now.

I just updated to docker-1.7.0-6.git74e7a7a.fc22.x86_64 which caused me to have to wipe out /var/lib/docker and set up some docker-storage like this:

$ cat /etc/sysconfig/docker-storage 
DOCKER_STORAGE_OPTIONS=-s devicemapper --storage-opt dm.fs=xfs --storage-opt dm.thinpooldev=/dev/mapper/vg_ksylvan--t420-docker--pool --storage-opt dm.use_deferred_removal=true

And

docker run --name=mongo -it docker.io/mongo

Worked, many many times, but now it does not. "docker images" only shows 2 image (docker.io/node and docker.io/mongo) and "docker ps -a" only shows the named mongo container, which is not starting up:

2015-07-15T22:28:35.892+0000 I JOURNAL  [initandlisten] 
2015-07-15T22:28:35.892+0000 E JOURNAL  [initandlisten] Insufficient free space for journal files
2015-07-15T22:28:35.892+0000 I JOURNAL  [initandlisten] Please make at least 3379MB available in /data/db/journal or use --smallfiles
2015-07-15T22:28:35.892+0000 I JOURNAL  [initandlisten] 

Any clues?

[ksylvan]

And docker info shows this:

$ docker info
Containers: 1
Images: 27
Storage Driver: devicemapper
 Pool Name: vg_ksylvan--t420-docker--pool
 Pool Blocksize: 524.3 kB
 Backing Filesystem: extfs
 Data file: 
 Metadata file: 
 Data Space Used: 1.475 GB
 Data Space Total: 53.59 GB
 Data Space Available: 52.11 GB
 Metadata Space Used: 548.9 kB
 Metadata Space Total: 469.8 MB
 Metadata Space Available: 469.2 MB
 Udev Sync Supported: true
 Deferred Removal Enabled: true
 Library Version: 1.02.93 (2015-01-30)
Execution Driver: native-0.2
Logging Driver: json-file
Kernel Version: 4.0.7-300.fc22.x86_64
Operating System: Fedora 22 (Twenty Two)
CPUs: 4
Total Memory: 15.56 GiB
Name: ksylvan-t420
ID: I4KU:N3YC:52JU:DBB3:QF5H:CFHH:Y547:VCDO:EEDD:XBW5:JIY6:IUCG
Username: kayvan
Registry: https://index.docker.io/v1/

Any help would be greatly appreciated.

[tammomueller]

@jareware Running into the same issue on OS X with boot2docker and official mongo image

[garystafford]

I have issues with mongodb image using Docker Machine with VirtualBox VM, using small files setting. Makes no difference.

[pencilcheck]

@yosifkit's answer in this issue docker-library/mongo#51 fixed it for me

[garystafford]

Cleaning up tagged images (#51) to create more space might be a band-aid, but it doesn't explain root cause.

[hongweiyi]

LGTM

[wattletimber]

Hi Guys am experiencing almost a similar problem. I really need a quick solution to this. I have dockerized mongodb and meteor and my mongodb image is 309mb but my container is increasing size with a 1 gig everyday. Hoe can i overcome this problem?

[agamrafaeli]

For me the problem was that I shut down my computer without shutting down the container properly.

When starting the container an internal problem happened with the mongo. I SSH'ed into the machine and deleted a lock file and then the container started all right for me.

[nathanmalishev]

My solution was to create a new docker-machine, with 30gb instead of 20gb

[patrickaroo]

I'm having the same issue on OSX, Docker Version 1.12.3-beta29.3 (13640), and tutum/mongodb, for what it's worth. The error on docker-compose up was:

db_1 ... ERROR: Insufficient free space for journal files
db_1 ... Please make at least 3379MB available in /data/db/journal or use --smallfiles

Since I'm using the new(ish) Docker app, instead of the old approach, it's a little unclear at first glance how I'll do the rebuild workaround, and restarting the Docker app isn't helping.

update: posting this in docker-library/mongo#51 ; please advise if this was not the right thing to do.

[msmichellegar]

Cleaning up my Docker data with these instructions fixed the issue, particularly removing unnecessary volumes.

[iammichiel]

It seems the problem is only present in the 3.0 version of MongoDB. Switching to a 3.1 or later version corrects the problem.

[marcusphi]

If someone stumbles on this issue more recently: Note that since mongo 3.2 the default storage engine is wiredTiger, which does not have smallfiles config, but it seems, as noted above, that mongo version 3.1+ solves the problem anyway.