Import/Export Rocket.chat data (messages and all configuration) via Administration

[lashae]

We have been using Rocket.chat as a deployed service on "Rocket.Chat Deploy" servers with the version 0.45.0.

Since we don't have access to the server and the terminal, we would like to download all Rocket.Chat configuration and message history via the Administration interface. This is a common requirement for anyone using hosted Rocket.chat just in case something happens... If export feature support an API, we can perform this exports (backups) in an unattended manner to our systems (to our S3 buckets).

Obviously, import screen on the Administration screen should support Rocket.Chat as well.

Since the consequences of not-having a backup is very dramatic , we kindly ask for a "high priority" "feature request" for this issue.

[Adirio]

This would not only allow to backup the chat, but also to make the data auditable when evidencies need to be given to a certain authority.

I would add user information to the message history and configuration.

[pierreozoux]

I understand your use case, but your hosting provider should be able to give you a way to perform unattended backups.

[lashae]

@pierreozoux Please check out the issue description, the issue is created mainly for deployed Rocket.chat instances. There is no hosting-provider, Rocket.chat is provided as a service by Rocket.chat itself. We use Rocket.chat as a subdomain on http://[subdomain].rocket.chat address.

On the other hand, hosting providers may or may not offer a way to create a custom backups, however having the ability to backup via Administration screen is a must-have feature. To create a custom backup, you need to fully-understand what and how-to backup all dependencies, and same is required to restore. Isn't it better to abstract backup strategy?

[pierreozoux]

@lashae yes, I read correctly. I'm myself a provider of rocket.chat as a service ( IndieHosters ) And we have a way for end users to backup there full data, unattended (data folders + mongo dump).

We are supporters of https://userdatamanifesto.org/.

But agreed, that a export/import via admin is also a nice feature.

[cutzenfriend]

+1

[alexleosumy]

+1

[pierreozoux]

@cutzenfriend @alexleosumy please, +1 comments are not needed anymore as you can 👍 the issue itself on top. When you do, you send a message to all participants which is not necessary.
(sorry for the spam for others, but we need to help people get up to speed :) )

[JSzaszvari]

related #758

Still don't think this should be a function of RocketChat but left for the database layer to take care of it.

If using a hosted solution it should be up to the host to provide access to mongo database backups for your instance, or a interface to get them using mongodump.

[ghost]

@pierreozoux can you share that script? (or a very simplified version related to extract the user's data from mongodb)

Ok not to be an option in user interface, but at least the script to do it as admin when a user request it (contact form, ticket, etc.). I think this is a very strange request for some users (these days, because you know, (joke) "you don't have nothing to hide", but when one user requests that and you are not providing this, it's very bad situation for user and the admin/org hosting the service (bad rating of service)

[pierreozoux]

@pedro-nonfree what kind of script are you looking for?

On my side, and for some client, I have a stupid thing that does the job:

• I dump the mongo on the client's folder: https://git.indie.host/meta/Rocket.Chat/blob/master/scripts/pre-backup
• then give a secret url to the client that allow him to download the full folder: https://github.com/indiehosters/wader

It is not perfect from a lot of perspective, but it is good enough for now.

[ghost]

Thanks for the links. I see, your client is an entire rocketchat installation, and you provide him entire mongo database. I would like to know what is the query to dump all conversations (in public and private) of a particular client of rocketchat to deliver to him. I see I'm in the wrong issue because this is for all data.

[lashae]

Don't take it offensive but recent discussions are off-topic in the "issue" context. Since we are interested in the implementation of the "issue" we cannot unsubscribe and every message is delivered to us as a notification via email.

Could you please kindly move the discussion to another thread/issue?

[bizzbyster]

Where does this issue stand? Our use case is very similar and could likely leverage this functionality. We'd like to be able to version control our server configuration. And then code review any changes made to it and then update the running server instance programmatically as part of our continuous integration/deployment (CI/CD) pipeline. I assume that if the Administration console supported Import/Export then we could implement that programmatically.

[bizzbyster]

I suppose we could try what @JSzaszvari suggests here #758. We'll see if that fits our need.

[ghost]

+1

[hupptechnologies]

Can't we have some setting in the administration panel where in we can set our S3 bucket credentials and bucket name and the entire chat backup would be updated automatically on regular intervals ?
This will eliminate even the need for exporting manually and then uploading to S3.
What do you think @lashae ?
We also use rocketchat for our internal use and would love to have this ability to backup users / config and chat history for various updated on some S3 compatible storage.

[bizzbyster]

This would be especially nice if it encrypted the data prior to reading/writing from/to S3.

[lashae]

I appreciate having an auto-backup procedure which is backed by cloud. However my personal preference is Google Cloud rather than AWS ;-)

[hupptechnologies]

Indeed. But having an option to auto backup is important here, rather than setting an external script to do the same.

[pierreozoux]

I don't think backup is part of the discussion here.
We are discussing about import/export, not backup/restore procedure. Please if you want to discuss about this, open a new thread.
(And in my opiniomn, it is the responsability of the operator to implement that. Not the nodejs code of RocketChat, or eventually the responsability of the package, like the helm package. )

[jeyca]

Still don't think this should be a function of RocketChat but left for the database layer to take care of it.

If using a hosted solution it should be up to the host to provide access to mongo database backups for your instance, or a interface to get them using mongodump.

I am currently in a situation where I administrate a Rocket Chat installation but do not have access to the server. The people who set up the Chat are not available at the moment and now I do not have any possibility to export the data from Rocket Chat.

Even if this is a special situation, I think having an export option should be accessible not only for server admins but also for App admins. It should even be possible for users to backup the rooms and messages they have access to.

[pierreozoux]

It is your responsibility as a client to choose your hoster properly, and think about this before bad things happen.
I don't want to "show off" but I am a hoster, and some of the clients asked an endpoint where they can pull backups to there disk. And I find it normal to do that.

For a list of nice hosters, please refer to:

• https://github.com/libresh/awesome-librehosters/
• https://chatons.org

(I'm not sure which hoster does provide RocketChat, but we are working on that :) https://github.com/libresh/catalogue/ )

[lashae]

@pierreozoux I would kindly ask for you to quit promoting your hosting service level. This is the second time, you interrupted the conversation with your promotion; we very well understood that you are a heroic Rocket.chat hosting provider, ok? Here, an issue (feature request) is created under an opensource project and we would like to limit the scope of discussion to the implementation details.

The issue here is neither selecting the best hosting provider nor whether it is the responsibility of the hosting provider. Any out-of-topic message lengthen the discussion and prevents conversation to come to a conclusion. Like your message this message is unfortunately delivered to all people contributed to this issue discussion and it unfortunately will cause waste of time.

By the way, I have been using RocketChat via official hosting provider (cloud.rocket.chat) and there lacks this import/export feature as well; they don't provide a backup solution either.

And lastly, the issue title is "Import/Export Rocket.chat data (messages and all configuration) via Administration", although some of the contributors commented as "backup"; import/export is not in deed backup. As a hosting provider, what you can do at your best effort is backing up the database or somefiles. If there exists some sort of version mismatch between the source and destination instances, your backup won't make any sense. Import/export tries to abstract this process and try to deliver best UX possible.

[pierreozoux]

Sorry if you misinterpreted my message. I'm not trying to promote my work. I'm totally aware that the target is not here, nobody here needs hosting as a service.
I just believe in hosting in a proper way like this:
https://userdatamanifesto.org/

I don't think it is a valid feature request, besides the need for a single user to export all messages because of GDPR.

The rest import/export is complicated to do it right, and it is easier to do with mongodump.
I don't think that Rocket.Chat will do this feature (but I might be wrong).
So good luck developing it :)

[marceloschmidt]

This kind of Import/Export should be done at database level for performance reasons. If you are running on our cloud infrastructure, you may ask for a database dump by mailing cloud@rocket.chat.

[jeremie-lesage]

@marceloschmidt Your support has not responded for 3 months. How can I ask for database dump ?