Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ability to delete chat history #2355

Closed
mikrobyte opened this issue Feb 26, 2016 · 62 comments · Fixed by #11236
Closed

Ability to delete chat history #2355

mikrobyte opened this issue Feb 26, 2016 · 62 comments · Fixed by #11236
Milestone

Comments

@mikrobyte
Copy link

Your Rocket.Chat version: 0.19.0

This is not an issue per se, but more of an enhancement request if possible.

It would be nice to have the ability to delete a chat's history, could be a room's history, or a 1 to 1 chat history. Totally, all messages displayed in a given room. Or delete all messages older than $DATE , or delete all messages between DATE1 and DATE2.

@mikrobyte
Copy link
Author

Could be that this feature is only available for server admins.

@engelgabriel engelgabriel added this to the Nice-to-Have milestone Feb 26, 2016
@sampaiodiego
Copy link
Member

you can delete rooms (again - check #2351) - look:
image

@mddvul22
Copy link

mddvul22 commented Jul 1, 2016

Thanks sampaiodiego, that is nice to know. However, it would be very nice to have the ability, as an admin, to auto expire messages. Perhaps a system-wide setting to forget messages older than a given time period. Or, perhaps to forget messages once a user logs out. Or perhaps best of all, to never keep messages on the server at all.

@nwildner
Copy link

Agree with this. Setting up auto-expiring channel history(like, daily, weekly) could be interesting in some cases. We are doing some integration of Rocket.Chat with Zabbix, and the channel that receives the notifications could be cleared from time to time to avoid long history and keeping daily notifications clean.

Meanwhile we can delete and recreate the channel from time to time, but we have to ask to all members to refresh their browsers since it keeps some of the old channel cache showing(and blocking new messages to be sent)

@FibreFoX
Copy link

I recently installed a fresh version of Rocket.Chat and played a bit. To not need to recreate a channel everytime, there IS some need to cleanup the thread. Is it possible to just clean previous messages from some message/time? (Especially automatization-chats would benefit from this)

@kvnblank
Copy link

With a MongoDB Query you can delete all messages in the whole db "less than" a specific date.

db.rocketchat_message.remove( { ts: { $lt: ISODate("2016-08-01") } } );

It is not the best solution, but in our company we had to find a workaround until the developers add some cleanup features.

@ulope
Copy link

ulope commented Nov 9, 2016

Another vote for this feature. We have a channel that logs a certain kind of exceptions from our sytems and those tend to be rather large - so the channel becomes very slow to load in short order.

Currently I'm deleting and recreating the channel on a regular basis, but that's not really a "solution"

@dtech4you
Copy link

is this something that is close to being usable the ability to remove all chat without having to remove the channel would be amazing or at least a way to clean the database...

@engelgabriel engelgabriel added the Feature: Request Requested Feature label Dec 26, 2016
@9joshua
Copy link

9joshua commented Jan 2, 2017

+1 for this feature

@cellisten
Copy link

+1 from me as well.

@seanloosier
Copy link

+1

The functionality I would like is slightly different, I'd like to clear DM's on logout

I could help test if needed

@ydixken
Copy link

ydixken commented Jan 9, 2017

+1

@antgel
Copy link
Contributor

antgel commented Jan 9, 2017

Dear +1ers, you can just click on the first post and add your thumbs-up there. :)

Regarding the feature, I think it would be extremely useful to be able to choose between 1) delete permanently, or 2) archive (perhaps to S3?) then delete.

@9joshua
Copy link

9joshua commented Jan 9, 2017

Dear +1ers, you can just click on the first post and add your thumbs-up there.

Yes but it wouldn't keep this thread alive nearly as well :)

@FibreFoX
Copy link

FibreFoX commented Jan 9, 2017

Yes but it wouldn't keep this thread alive nearly as well :)
Thats no real reason, because it makes this thread just noisy.

As this project is pull-request friendly, maybe someone can create a PR instead of just +1-ing? I'm currently not (yet) able to develop, but I'm preparing to. Maybe I will fast enough to develop this feature (but I doubt it, because I'm not used to develop meteor-driven apps)

@seanloosier
Copy link

Added some thumbs up per suggestions :)

@FibreFoX
Copy link

FibreFoX commented Jan 10, 2017

After digging around a bit, this feature seems to already exist:
https://rocket.chat/docs/developer-guides/rest-api/channels/cleanhistory

As far as I understood, this only is a missing button, am I correct?

EDIT: there is even some permission already: clean-channel-history

@mddvul22
Copy link

That seems to be only for channels, right? I believe this is for deleting other stuff, as well.

@FibreFoX
Copy link

@mddvul22 looking at the sourcecode, it only is for channels:

RocketChat.API.v1.addRoute('channels.cleanHistory', { authRequired: true }, {

@sinteur
Copy link

sinteur commented Feb 1, 2017

It does not work for private groups which you temporarily change to public channel for a minute, just to be able to use the api - so either this functionality needs to be duplicated for private groups, or it needs to be fixed so that the workaround works

@chrisjd20
Copy link

I use the solution that that KKvin came up with but expanded upon it to clear the uploaded files that were part of the 30 day threshold as well. Not elegant but it gets the job done. Can be seen here:
https://github.com/chrisjd20/rocket_chat_history_clear

@timmc
Copy link

timmc commented Mar 12, 2017

Just to cross-link: If you want retention policy rather than manual deletion, There's An Issue For That: #795

@chrisjd20
Copy link

chrisjd20 commented Mar 12, 2017 via email

@WebSavvyDude
Copy link

any progress on this?

@WebSavvyDude
Copy link

just a reply for the issue in general. If we can make something in the admin panel to delete old message history.

@thepowerprocess
Copy link

Based on my experience, deleting messages via REST API causes the client side to update automatically, so their is no issue with cached messages.

@graywolf336
Copy link
Contributor

@MarcosSpessatto Right now we currently have channel.cleanHistory which utilities a real time method. It would be super easy to deprecate this endpoint and instead move over to rooms.cleanHistory, that way any room is viable and not just the channels. Less than fifteen minutes of work is required as it's that simple. Just be sure to depreciate the channels endpoint.

RocketChat.API.v1.addRoute('channels.cleanHistory', { authRequired: true }, {
post() {
const findResult = findChannelByIdOrName({ params: this.requestParams() });
if (!this.bodyParams.latest) {
return RocketChat.API.v1.failure('Body parameter "latest" is required.');
}
if (!this.bodyParams.oldest) {
return RocketChat.API.v1.failure('Body parameter "oldest" is required.');
}
const latest = new Date(this.bodyParams.latest);
const oldest = new Date(this.bodyParams.oldest);
let inclusive = false;
if (typeof this.bodyParams.inclusive !== 'undefined') {
inclusive = this.bodyParams.inclusive;
}
Meteor.runAsUser(this.userId, () => {
Meteor.call('cleanChannelHistory', { roomId: findResult._id, latest, oldest, inclusive });
});
return RocketChat.API.v1.success();
}
});

@HackyPenguin
Copy link

We currently have the same issue in requiring a cleanup I ended up writing a python based script very similar in logic to the php script above however as we have to delete literally millions of messages and each of those requiring a seperate api call which took around a second to complete with the python request library this was impractical. and seemed to cause a massive load over an extended period of time. I am currently rewriting in hope of changing private groups to public and using the channels.cleanhistoryyes endpoint. however this is hacky I would much prefer a rooms.cleanhistory endpoint for bulk message deletion.

@HackyPenguin
Copy link

In case anyone else also needs this here is a script which can delete both public and private rooms using the api. https://github.com/HackyPenguin/rocketchat_cleanup/tree/master

@ariekogan
Copy link

Same requirement here ...
We need a channel / room that we can set an expiration period for the messages in this channel/room.

@rsimai
Copy link

rsimai commented Apr 24, 2018

To second this feature, we also have that requirement not for technical but legal reasons.

Background: in Germany (and probably other countries) data protection law requires to define a maximum storage time for data. That means we need a reliable way to set an expiry date (e.g. "after 5 years") on all content in all channels.

Ideally there's a global setting as well as by channel settings, to allow a faster purge for automation channels.

@FibreFoX
Copy link

@rsimai not only germany, with the upcoming GDPR (General Data Protection Regulation) the whole EU is having "new" data protection laws, making the whole history-thing a bit more painful than it might have to.

@waterboo
Copy link

waterboo commented May 3, 2018

@FibreFoX "upcoming"... as in... the 25th of May 2018! As far as I understand GDPR we will have to have the ability to automatically delete messages after a specified time or upon user request.

@marceloschmidt marceloschmidt added Feature: Planned Planned Feature and removed Feature: Request Requested Feature labels May 10, 2018
@marceloschmidt marceloschmidt modified the milestones: Long-term, 0.67.0 May 10, 2018
@engelgabriel engelgabriel mentioned this issue May 10, 2018
3 tasks
@eszeus
Copy link

eszeus commented Jun 18, 2018

Is there anything new to this, or information about when it will be implemented?
We are having the same issue here in our office, the new EU Law for data protection is really painful.

Did anyone automated the channel history deleting process and can share it with us?

@vynmera
Copy link
Contributor

vynmera commented Jun 18, 2018

@eszeus There is the REST API call rooms.cleanHistory. I might make a PR to implement a GUI for it, in the channel buttons though, to make this easier (and possibly make it automatically stage it so you can tell the process is happening, as cleaning a large channel may take ages)

@eszeus
Copy link

eszeus commented Jun 19, 2018

@vynmera Any Automated process would be awesome! Specially with a GUI. I am not an expert with APIs and such, sadly...

@vynmera
Copy link
Contributor

vynmera commented Jun 25, 2018

I'm making it happen! ---> #11236

@xmovu
Copy link

xmovu commented Jun 25, 2018

@vynmera that's pure awesomeness! How can I deploy it? Has it been merged to master? Don't know much about git!

@vynmera
Copy link
Contributor

vynmera commented Jun 25, 2018

@wdimd You can clone my fork git clone https://github.com/vynmera/Rocket.Chat.git, go to it cd Rocket.Chat/, then go to my branch git checkout purge, and deploy it with meteor npm start. For more info on how to deploy a development version, check this out: https://rocket.chat/docs/developer-guides/quick-start/

Note that it's a WIP build, don't expect everything to work correctly (or at all). I highly advise you to not use it in production :)

When I update anything, you can git pull to download the newest version.

@xmovu
Copy link

xmovu commented Jun 25, 2018

Thanks @vynmera, keep rocking!!!

@eszeus
Copy link

eszeus commented Jun 27, 2018

@vynmera also deployable via snap installation? :D

@vynmera
Copy link
Contributor

vynmera commented Jun 27, 2018

@eszeus I'm afraid not, it's not done yet ~ but when it is, you can start using it once it is on Rocket.Chat's snap (probably next month, at 0.67.0)!

@eszeus
Copy link

eszeus commented Jun 27, 2018

@vynmera so nice! Thanks! Sounds great to me :D Thank you for this work. Awesome, and neccessary xD

ggazzo pushed a commit that referenced this issue Jul 20, 2018
Closes #6749
Closes #8321
Closes #9374
Closes #2700
Closes #2639
Closes #2355 
Closes #1861
Closes #8757
Closes #7228
Closes #10870
Closes #6193 
Closes #11299
Closes #11468
Closes #9317
Closes #11300 (will incorporate a fix to this PR's issue)
Closes #11046 (will incorporate a fix to this PR's issue)
Contributes to #5944 
Contributes to #11475
_...and possibly more!_

This PR makes deleting messages (automatically and manually) a lot easier on Rocket.Chat.

- [X] Implement a bulk message deletion notification, to quickly push large message deletions to users without reload
  - [X] Use it in `rooms.cleanHistory`
  - [X] Use it in user deletions
- [X] Completely remove cleanChannelHistory as required by v0.67
  - [X] Remove server method `cleanChannelHistory`
  - [X] Remove REST API `channels.cleanHistory`
- [x] Implement a sidebar option to clean history
  - [x] Basic implementation
  - [x] Allow excluding pinned messages
  - [x] Allow attachment-only mode
  - [x] Allow specifying user(s) to narrow down to
    - [x] Also update REST API
    - [x] Also update docs
  - [x] Break the deletion into multiple different requests, so the client can keep track of progress
  - [x] Clear animation / progress bar for deleting
- [x] Retention policy
  - [X] Global, set by admin
    - [X] Global timer that runs every second and deletes messages over the set limit
      - [X] Can change its timer's resolution to prevent insane CPU overhead
    - [X] Admin can decide what room types to target (channels, groups and/or DMs)
    - [X] Allow excluding pinned messages
    - [X] Allow attachment-only mode
  - [x] Per-channel, set by those with a new permission
    - [x] Disabled when master switch off
    - [x] Set in channel info
    - [x] Can override global policy with a switch that requires `edit-privileged-setting`
    - [x] Allow excluding pinned messages
    - [x] Allow attachment-only mode
    - [x] Uses same global timer for cleanup
  - [X] Message at start of channel history / in channel info if there is a retention policy set
  - [x] Message in channel info if there is a retention policy set on that channel specifically
- [X] Make cleaning history also delete files (completely!)
  - [X] Manual purging
  - [X] Automatic purging
- [x] Make other deletions also delete files
  - [x] User deletion
    - [X] Own messages
    - [x] DMs with them's partner messages
  - [x] Room deletion
- [x] Cleanup
- [x] Finish related [docs](https://github.com/RocketChat/docs/pull/815)
- [x] Link to the docs in the settings

Please suggest any cool changes/additions! Any support is greatly appreciated.

**Breaking change:** This PR removes REST API endpoint `channels.cleanHistory` and Meteor callable `cleanChannelHistory` as per the protocol specified for them.

![bzzzzzzzz](https://user-images.githubusercontent.com/39674991/41799087-56d1dea0-7670-11e8-94c0-bc534b1f832d.png)
@theorenck theorenck modified the milestones: 0.67.0, 0.68.0 Jul 20, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.