New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Lack of mongo resilience after unplanned shutdown. #6

Open
wifimaclet opened this Issue Feb 21, 2018 · 5 comments

Comments

Projects
None yet
3 participants
@wifimaclet
Copy link

wifimaclet commented Feb 21, 2018

Using stable Docker and latest docker-compose, my laptop overheated and emergency shut down kicked in due to reasons unrelated to wekan. After cooling down and powering back on, the containers started without error messages, but with an unclean shutdown warning from mongo. The boards were usable, but login was not, I was getting "Login Forbidden".

After some research (four hours divided in multiple days), I found the solution in Stack Overflow, apparently mongo doesn't self-repair when starting unless it has journaling enabled. I do not know how to enable journaling in mongo, but a docker-compose down; docker-compose run wekandb mongod --repair; docker-compose up fixed my issues.

@xet7

This comment has been minimized.

Copy link
Member

xet7 commented Feb 21, 2018

@wifimaclet

Is it safe to include that option always? What changes should be made to this repo's docker-compose.yml file?

@wifimaclet

This comment has been minimized.

Copy link
Author

wifimaclet commented Feb 21, 2018

Journaling is safe for production, yet I don't know how to activate it. --repair, though, is not safe for mongod clusters, yet I could not find any other safety information regarding it.

@xet7

This comment has been minimized.

Copy link
Member

xet7 commented Feb 21, 2018

@wifimaclet

If you find how to add it, you can add documentation about it as pull request to this repo master branch docker-compose.yml file. If it's only safe to use when repairing, you can leave it uncommented.

@ofekp

This comment has been minimized.

Copy link

ofekp commented Sep 24, 2018

For anyone interested, this is how I go about doing it:

version: "2"
services:
  mongo:
    container_name: mongo
    restart: on-failure:10
    image: mongo
    environment:
      - MONGO_INITDB_ROOT_USERNAME=<dbuser>
      - MONGO_INITDB_ROOT_PASSWORD=<dbpass>
      - MONGO_INITDB_DATABASE=db
    volumes:
      - ./data:/data/db
    ports:
      - "27017:27017"
    command: bash -c "mongod --repair && mongod"

mongod --repair will repair the issue with the lock file, then mongod will start mongo regularly, this is needed since the repair also shuts down mongo.

This way I do not change the docker-compose up command to start my service.

@xet7

This comment has been minimized.

Copy link
Member

xet7 commented Sep 24, 2018

@ofekp

Thanks! I added to wiki link to this post.
https://github.com/wekan/wekan/wiki#Repair

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment