[wip] purge messages from server after N days #595
Conversation
|
I guess that there will be an option to purge messages older than 0 days, so that all messages instead of moving to DeltaChat folder are purged directly and self-bcc are not sent at all, this will open possibilities for people more interested in good multidevice support than in syncing.
|
|
Hi @r10s What do you think? |
|
we need messages to be permanently deleted that is the whole point of this feature, the messages that will be deleted are the messages on DeltaChat folder, so what you suggests is to move messages to the Thrash folder instead of to the DeltaChat folder which is not much useful since the messages still remains on the server and we need to free space.
|
|
Is there even an IMAP command which really deletes messages circumventing the trash? |
|
@STPKITT well you actually flag them as /Deleted and then issue expunge command, or something like that. |
|
Please implement this to be the server-limits and -cleanup part of the "auto-delete" background task from https://support.delta.chat/t/reduce-space-on-internal-storage-disk-full/132/24 (also keeping in mind its later extension to local-limits and -cleanup) The -limits related details from #120 contain a nice storage space condition:
|
|
With "location streaming" comes a lot more mails which flooded the mailbox. So delete older messages becomes more important. |
|
The problem of many meta-data messages mingled within regular messages won't get solved by a history limit. Looks like #112 remains a different issue, that might best be solved with #644, using a "first see, first delete" approach that stores the updated state for the other clients using message flags or state messages. |
|
@testbird I could set it e.g. to 30 days. Time enough to sync my clients. People with really less storage (e.g. @adbenitez) and no need for sync, could set it e.g. to 1 day or so. So purge older messages would definitely help. |
|
webratte actually I was expecting a set to 0 days where the messages are deleted instead of moving them after they are downloaded and the self-bcc copy isn't even send :) really I don't want them on the server at all
…On April 21, 2019 1:12:23 AM GMT-04:00, webratte ***@***.***> wrote:
@testbird
I don't agree.
A history limit would help to save server storage.
I could set it e.g. to 30 days. Time enough to sync my clients.
People with really less storage (e.g. @adbenitez) and no need for sync,
could set it e.g. to 1 day or so.
So purge older messages would definitely help.
--
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub:
#595 (comment)
--
Sent from my Android device with K-9 Mail. Please excuse my brevity.
|
|
and when I say "me" I am talking of the Cuban use case of curse, not my particular preference it is just that we need to use 50 MB for a month with care
…On April 21, 2019 1:12:23 AM GMT-04:00, webratte ***@***.***> wrote:
@testbird
I don't agree.
A history limit would help to save server storage.
I could set it e.g. to 30 days. Time enough to sync my clients.
People with really less storage (e.g. @adbenitez) and no need for sync,
could set it e.g. to 1 day or so.
So purge older messages would definitely help.
--
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub:
#595 (comment)
--
Sent from my Android device with K-9 Mail. Please excuse my brevity.
|
That's not the problem, and not what I wrote. |
|
I think this can refined in a later version.
As a first step it would be enough to delete mails older then "n" days.
Or maybe directly after download if no sync is needed.
|
|
Yeah, #112 is a separate issue, that's what I meant (besides that actually quota based deletes will be needed to guarantee free space, especially with small servers). #112 is not solved with this issue, contrary to the suggestion in the report (and #112 is actually better solved by deleting receipts with syncing feature #644 rather than only with the corresponding message). |
|
Wait a second, this issue is about server space, right? Then why would one want to delete messages after N days, instead of limiting the number of messages? Maybe have a look at this (updated) discussion post (again), |
|
testbird, people want to delete based on time because in general the older a message is, the less important it is for an user, and the less they will remember it even exists |
Remember this is a server issue. It's like this: And people also usually only want, and only need to deal with simple options to adjust the "history lengh" kept in chats. But this is NOT the "Max. number of messages kept on server", for which there should usually be no need for the user to change it. The useful knobs for the user to shorten the list of messages in chats is rather "Limit number of (archived) messages (per chat)" or "Limit age of (archived) messages (per chat)", on the device. The latter would , however, also completely delete important but low traffic chats. Note that it is actually not required to let user configure a history limit. (There are users with huge server and device strorage.) What's needed, however, is an auto-delete-server and auto-delete-client feature to avoid disk full errors, and avoid having to ship with a default history limit that would make all large server's storage space useless. History limits should still be easily configured though, if desired, as a personal preference: If you review the options (I improved the sorting and wording.), I think the server setting "Max. number of messages kept on server, and thus synced across devices (per chat)" (topic of this issue) would be a really advanced "admin" setting, needed only for the mentioned special use-cases. And it makes sense for it to be an integer value, in order to make it easier for multiple watching clients to count and sync the n messages on the server, and detect those that expirie at the end of the list of messages on the server automatically. The user does not have to notice anything about this and only sees the total length of the chats. |
|
I don't understand you.
There are much very important things to do.
Why should the devs start complicated as you mentioned?
This issue is important.
But a relatively easily start (so I think, I can't coding) could be:
* Check the deltachat folder every e.g. monday for messages older then 6 month as a user preference
* Now delete this older messages without to have to check which chat is involved.
Of course, this can be defined if the devs onetime will have nothing more important things to do 😉
|
|
If using a period of days for expiration on the server, a second client can not know the amount of days to keep a message from seeing just the list of chat messages on the server, and can't distinguish between a delete to sync (delete locally) and a delete to keep (archive locally). So, periodical expiration will not work well for multiple devices, on some syncing devices, nothing may be kept (i.e. enter the local archive) even if configured. |
|
Sorry, I don't know what you mean...
|
That will look strangely erratic to users, not help in a week with a lot of spam messages, not free up enogh space, ... and has problems if device is off one day. Simplest: do the check on every client with every received message.
We're talking about the server here, and keeping a Trashbin is useless to free up space. Users will complain: I come back from vacation and deltachat deleted all my important messages (from the server, now not syncing anymore) just because friend/group X had sent me a lot (or large) messages. |
|
Avoid sure problems when a better way is known. |
|
And again, considering the broader picture https://support.delta.chat/t/reduce-space-on-internal-storage-disk-full/132/24 this issue can be seen as a first implementation of "auto-delete-server". |
We talking about DC (the App) will check the server and delete the older messages.
Only if they was more then 6 month in hollydays (for my example) |
See, particular examples are not working in general. Other users with few server space would have to reduce this purge setting a lot. And everything must still work. Half baked "features" can be detrimental. |
Of course, mails should not deleted before it's downloaded. |
|
I also think that instead of dreaming with a "perfect" method, we start with something simple that "works", we are in a situation with 100MB of server space, users run out of storage and they blame Delta Chat, because other email-chatting apps don't have this "error" so we need something that just delete the messages after they are downloaded to DC is data base, so even starting with a simple option like that (keep on server or remove them after download) will be enough for an start, of curse deleting based in time would be also something good to have, my point here testbird: |
|
I don't think anybody said to delete without at least the first client having downloaded the message. But the problem comes with devices that sync later, especially if coming back after a longer period.
Yes, sure. I am all for a very simple, first auto-delete-server, even if starting by only covering the "fallback", i.e. for the case of servers without quota support (limit to some default number of max. messages on the server). My point is just that a time period based limit for server deletes does not work for multi-client sync. (It's a mistake to avoid.) A total max. number of chat messages to keep on the server, however, should work. It is a not-user-facing technical server related setting that can also work for syncing later I think. For limiting the chat history that the users keep and see on the device, a much more useful option for the user can be a simple separate, and user-visible "max. number of messages per chat" (relevant for the auto-delete-device). |
|
I think we should stop at this point our discussion and let the experts (not you and not I but the devs) do what they think it's a possible way.
Isn't it?
|
|
OK. It should work for a single device and multiple devices because:
So this should work without requiring any separate sync and coordination data on the server, and no traffic overhead. |
|
Should this issue reopened in rust-core? |
this is not an issue, this is the unfinished try to fix an issue. the belonging issue is still open and already moved to core-rust: deltachat/deltachat-core-rust#512 |
todo:
to make things as safe as possible, we'll only purge message from the DeltaChat folder or, if moving is disabled, from the INBOX by a positive list (the known messages in the database)
targets #164 and maybe #112