Reusing cassandra image with data preloaded from a docker commit

[mpeters]

I am trying to setup a developer environment with a cassandra container from an image that is based on this one, but with data already loaded. The reason is because loading the test data is complicated and relies on 3rd party components and other running containers, so it's just something that can just be done in a Dockerfile.

So we have a running container (just a single container, no cluster) with the data prelaoded and we save that as an image with docker commit. This works just fine. But when we try to use that image later it fails with an error like this:

INFO  19:58:33 OutboundTcpConnection using coalescing strategy DISABLED
ERROR 19:59:04 Exception encountered during startup
java.lang.RuntimeException: Unable to gossip with any seeds
        at org.apache.cassandra.gms.Gossiper.doShadowRound(Gossiper.java:1307) ~[apache-cassandra-2.1.7.jar:2.1.7]
        at org.apache.cassandra.service.StorageService.checkForEndpointCollision(StorageService.java:533) ~[apache-cassandra-2.1.7.jar:2.1.7]
        at org.apache.cassandra.service.StorageService.prepareToJoin(StorageService.java:777) ~[apache-cassandra-2.1.7.jar:2.1.7]
        at org.apache.cassandra.service.StorageService.initServer(StorageService.java:714) ~[apache-cassandra-2.1.7.jar:2.1.7]
        at org.apache.cassandra.service.StorageService.initServer(StorageService.java:605) ~[apache-cassandra-2.1.7.jar:2.1.7]
        at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:378) [apache-cassandra-2.1.7.jar:2.1.7]
        at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:524) [apache-cassandra-2.1.7.jar:2.1.7]
        at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:613) [apache-cassandra-2.1.7.jar:2.1.7]

So the seed value (which contains the IP address the container had when it was preloaded before the commit) is wrong for the new container because the IP address has changed.

So how can we work around this? Can we tell cassandra to ignore it's previous seeds when it starts up? Can we somehow clear the IP/seeds of the previous container before we commit it to a new image? Something else?

Thanks

[yosifkit]

As long as you've moved where the data is stored, the commit should work fine, because the VOLUME defined in the Dockerfile would prevent it from being saved in the docker commit if left in the default path.

You'll probably need to update the cassandra.yaml on startup with new seeds and possibly other configuration like we do in the current entrypoint script.

[mpeters]

@yosifkit I'm not sure I understand. I do want the data to be part of the commit. The image preloaded with the data is what I'm trying to distribute to my team.

[yosifkit]

Using the image "as is", the database files will not be saved with a docker commit since they reside in a VOLUME in /var/lib/cassandra (see also: understanding volumes). You will need to configure cassandra to store the data in any place besides /var/lib/cassandra, like /data or something, so that the contents will be saved in a docker commit.

[mpeters]

Thanks, that did it. I wrapped this image with a layer that sets the data directory to /data/cassandra. I also set CASSANDRA_SEEDS to be the hostname of the container when we do the commit. Everything works fine.