Export from Wekan Sandstorm grain .zip file

Lauri Ojansivu edited this page Jan 28, 2019 · 11 revisions

TODO: Feature Request: command line utility to convert between Standalone Wekan MongoDB database, and Sandstorm grain .zip file

This is useful for example if you get Board not found error

Using Ubuntu 18.04 or Mint 19.1 or Ubuntu for Windows Subsystem for Linux:

1) Install required packages:

sudo apt-get install zip unzip p7zip-full wget

2) If you don't have Meteor 1.6.0.1 installed, download it to your home directory, and unarchive it

cd
wget https://releases.wekan.team/meteor-repair.7z
7z x meteor-repair.7z

3) Install MongoDB 3.2.22

https://docs.mongodb.com/v3.2/tutorial/install-mongodb-on-ubuntu/

4) Download Wekan grain

Use Sandstorm arrow down button to download Wekan grain in .zip file to your home directory.

5) Unzip downloaded file

unzip "YOUR-GRAIN-NAME-HERE.zip"

6) Repair database, if it does not open

.meteor/packages/meteor-tool/1.6.0_1/mt-os.linux.x86_64/dev_bundle/mongodb/bin/mongod --dbpath "./YOUR-GRAIN-NAME-HERE/data/wiredTigerDb" --repair

7) (Note to self about future versions of Meteor and Wekan)

Currently Wekan uses Meteor 1.6.0.1. In .meteor/packages/meteor-tool/ directory is also some 1.8.x version for upcoming Wekan version using Meteor 1.8.x. This .meteor directory has been generated by installing meteor from www.meteor.com and then upgrading some existing or new app with for example meteor update --release 1.6.0.1 so that correct version of Meteor is downloaded.

8) Stop MongoDB

sudo systemctl stop mongod

9) Copy repaired database to MongoDB raw files and start MongoDB

sudo su
cd /var/lib
mv mongodb mongodb-original
cp -pR "/home/YOUR-USERNAME/YOUR-GRAIN-NAME-HERE/data/wiredTigerDb" .
mv wiredTigerDb mongodb
chown mongodb:mongodb mongodb -R
systemctl start mongod
exit

10) Browse database with MongoDB CLI

mongo
show dbs
use meteor
show collections
db.users.find();
exit

11a) Dump database to MongoDB dump format

mongodump --db meteor

11b) Dump database to JSON text files

Save this to dump-json.sh and then chmod +x dump-json.sh && ./dump.sh meteor filesdir

Script is originally from this Rocket.Chat issue

#!/bin/bash

if [ ! $1 ]; then
        echo " Example of use: $0 database_name [dir_to_store]"
        exit 1
fi
db=$1
out_dir=$2
if [ ! $out_dir ]; then
        out_dir="./"
else
        mkdir -p $out_dir
fi

tmp_file="fadlfhsdofheinwvw.js"
echo "print('_ ' + db.getCollectionNames())" > $tmp_file
cols=`mongo $db $tmp_file | grep '_' | awk '{print $2}' | tr ',' ' '`
for c in $cols
do
    mongoexport -d $db -c $c -o "$out_dir/exp_${db}_${c}.json"
done
rm $tmp_file

It's also possible to restore JSON files like described at stackoverflow this way, but because userIDs etc can be different, there is no mapping to other users, so you may need to change JSON files some way:

for file in *.json; do c=${file#*exp_yourdbname_}; c=${c%.json}; mongoimport --db yourdbname --collection "${c}" --file "${file}"; done

12) Save attachments to files

Download NoSQLBooster GUI webpage

If you download AppImage for Linux, run it with:

chmod +x nosqlbooster4mongo*.AppImage
./nosqlbooster... (full filename here)

It can install NoSQLbooster to Linux Menu/Programming/NoSQLBooster for MongoDB.

With it connect to localhost:27017

Double click cfs_gridfs.attachments.files and right click filename and Download file.

Wekan logo

13) Optional: Restore

a) Restore data to Standalone Wekan Snap

b) Figure out how to make Sandstorm grain .zip file


OLD CONTENT, TO BE CLEANED UP SOMETIME

5) Copy database files from unzipped Wekan grain to MongoDB

In .zip file database files are in directory data/wiredTigerDb/

In Wekan Snap, database files are in directory /var/snap/wekan/common - if you have other data there, rename common directory to other name first.

Snap Backup and Restore

Docker Backup and Restore

6) Change database file permissions to root user

In Snap:

sudo chown root:root /var/snap/wekan/common -R

7) Start MongoDB

In Snap:

sudo snap start wekan.mongodb

8) Or, if you are brave, also start standalone wekan

Try does Standalone Wekan work with that database

sudo snap start wekan.mongodb
sudo snap start wekan.wekan

or both wekan and database at once

sudo snap start wekan

9) To see MongoDB data, install Robo3T GUI

https://robomongo.org

Wekan Snap: Connect it to address localhost:27019

Wekan Docker: If you have MongoDB exposed to outside Docker, Connect with Robo3T to address localhost:27017. Otherwise you first need to copy files from wekan-db Docker container to outside of Docker container, and restore files to local separately installed MongoDB database v3.2.20 or newer

Snap Backup and Restore

Docker Backup and Restore

10) Browse data in Robo 3T

  • On left, double click cards, checklists etc collections/tables to see their contents
  • Currently default view to see is tree view. To see table view or JSON view, click small buttons above that white data area view display at near right border.

11) If you don't find data in Robo 3T, use hex editor GUI

In Linux, you can install Hex Editor for example with command:

sudo apt install ghex

or

sudo yum install ghex

Then it's usually in Linux desktop at Menu / Development / GHex. Or you can start it in command line by writing ghex

You can open files from your unzipped Wekan Sandstorm grain directory data/wiredTigerDb/ to see if there is still some data that is not yet overwritten with other data.

12) Moving data from Sandstorm to Standalone Wekan like Snap or Docker

12.1 Compare databases of Sandstorm and Docker Wekan with mongo GUI https://nosqlbooster.com/

12.2 Create new user to Standalone Wekan

12.3 Copy new user bcrypt password structure created bcrypt encrypted password from Sandstorm to Docker wekan structure. It's at users collection/table. Also add isAdmin=true boolean.

12.4 Login

12.5 Here is info about script how to save all from MongoDB to .JSON textfiles https://forums.rocket.chat/t/big-issue-with-custom-javascript/261/4 and most likely how to get them back to mongodb so just copying all of those boards JSON files and restoring to MongoDB database would work and editing JSON files with any plain text editor. For big files you can try JEdit.

12.6 Here is how to backup and restore mongodb in docker, snap etc https://github.com/wekan/wekan/wiki/Backup

12.7 All Wekan Docker settings are in this textfile https://raw.githubusercontent.com/wekan/wekan/devel/docker-compose-build.yml

12.8 Database name in Sandstorm is meteor, and in docker is wekan, so here is how to restore with different database name https://stackoverflow.com/questions/36321899/mongorestore-to-a-different-database

Azure links

https://docs.microsoft.com/en-us/azure/container-service/dcos-swarm/container-service-swarm-mode-walkthrough

https://docs.docker.com/docker-cloud/cloud-swarm/create-cloud-swarm-azure/

https://docs.atlas.mongodb.com/reference/microsoft-azure/

Wekan

General

Support priorities for new features and bugfixes

  1. Commercial Support and Bounties
  2. Community Support
  3. Debugging

Security

Backup

Repair

Features

Email

Logs and Stats

Migrating

Settings

Download

Webservers

REST API Docs

REST API issue

REST API client code

Webhooks

Case Studies

Development

Issues

Clone this wiki locally
You can’t perform that action at this time.
You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session.
Press h to open a hovercard with more details.