Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Limit retained history (continued from c-core) #1277

Closed
csb0730 opened this issue Feb 15, 2020 · 19 comments
Closed

Limit retained history (continued from c-core) #1277

csb0730 opened this issue Feb 15, 2020 · 19 comments

Comments

@csb0730
Copy link

@csb0730 csb0730 commented Feb 15, 2020

This issue (open) deltachat/deltachat-core#244 should be continued here because it is necessary for long term use of DC.

Here the first entry of that issue from @r10s:


to avoid taking up all device "disk"-memory, there should be an option to limit the retained history.

  • a simple, initial approach would be just to set a max. number of messages allowed in a chat; if more messages arrive, old are deleted (from the device, not from the server)
    EDIT: the "contact requests" are also a "chat" in this sense
  • compared to using a fixed time, this would have the advantage that rarely used chats are not affected.
  • "starred" messages are not deleted

@searoso

This comment has been minimized.

Copy link

@searoso searoso commented Feb 20, 2020

Is there any relations with search? People could go mad when search won't give them old results. So it would be nice to have a related issue on server-side search.

The question is how to show results of server-side query. So we will come to dynamic loading of needed messages and clearing stale ones from the device.

P.s. On the topic fixed time VS max. number I would rather propose to have a limit of locally stored messages which can be set in settings (with default of 512 MB) and cleat oldest unstarred messages to keep the limit.

@r10s

This comment has been minimized.

Copy link
Member

@r10s r10s commented Feb 20, 2020

i do not think we will for got server-side search anytime soon. this has too many drawbacks wrt speed and encryption, complexity, maintanance.

the data are stored mainly on the device.

ux-wise, this is no issue, when deletion on device and server is synchronized, one thing to keep in mind on your suggestion in #1299 (comment) - not offering deletion on device but not on server would probably make things more complicated and easier to misundestand.

but anyway, this can may be better discussed in the forum at https://forum.delta.chat

@searoso

This comment has been minimized.

Copy link

@searoso searoso commented Feb 22, 2020

@r10s

This comment has been minimized.

Copy link
Member

@r10s r10s commented Feb 23, 2020

closing in favor to the more concrete #1309 - please note that deltachat/deltachat-core#244 is from 2018 and conditions and requirements have changed since then. eg. one goal is to have an ephemeral mode, where seized devices do not contain arbitrary old information, this is not doable with defining "old" by the number of messages.

also, the average amount of disk-memory available on devices has probably increased since then - the usecase 'avoid taking up all device "disk"-memory' is probably still there, but as an initial approach, things have shifted.

@r10s r10s closed this Feb 23, 2020
@csb0730

This comment has been minimized.

Copy link
Author

@csb0730 csb0730 commented Feb 23, 2020

closing in favor to the more concrete #1309 - please note that deltachat/deltachat-core#244 is from 2018 and conditions and requirements have changed since then

Hey! (Basic) requirements have not changed since then. Why they should? Nothing has been done since then relating that issue! Even more worse, workaround to jump to first message in a chat isn't existing any more.

I find it not ok to simply state now "requirements changed"!
We discussed many pros and cons in old issue and found, that simply deleting messages after N-days at device is not ok (one hint: low running chats!). I don't want to delete these simply after N days!

Conclusion: IMHO requirements have not changed, but maybe they need to be extended!

@r10s

This comment has been minimized.

Copy link
Member

@r10s r10s commented Feb 23, 2020

Nothing has been done since then relating that issue!

this is the reason i was closing this issue - no one was really working on it the last time. this issue tracker is about bugs and features actually implemented by some devs. for feature suggestions, please use https://forum.delta.chat

in contrast to this issue, there is #1309 that gets actually implemented, for reasons mentioned above.

Conclusion: IMHO requirements have not changed, but maybe they need to be extended!

that might be true - but it also has to be done. i mean, the original issue has not been implemented since years, so extending it will probably not help on getting it implemented faster.

see #1309 as a start, maybe some dev will enhance the feature to different needs at some point :)

@r10s

This comment has been minimized.

Copy link
Member

@r10s r10s commented Feb 24, 2020

btw. i am meanwhile also no longer convinced that the idea "delete all but N messages" is a good way to cleanup your device at all - even though i suggested that myself ;)

maybe sth. as deleting the largest messages first is better - deleting one video could make space for thousands of text messages. but also this is not that easy to implement and requires probably even more ui. and feature like that are in competition with tons of other things to do on 5 platforms - so, nothing for now :)

@r10s r10s added the UX-enhancement label Feb 24, 2020
@searoso

This comment has been minimized.

Copy link

@searoso searoso commented Feb 25, 2020

@r10s

maybe sth. as deleting the largest messages first is better - deleting one video could make space for thousands of text messages. but also this is not that easy to implement and requires probably even more ui. and feature like that are in competition with tons of other things to do on 5 platforms - so, nothing for now :)

that's smth I was thinking too, but recent video could be really useful
on the long run it lead to a lot of small messages kept forever and files deleted immediately (heard this anecdote on sand & stones in the jar?)
so the final metric should end up including both parameters
and oldest first look quite simple and effective as initial one

@csb0730

This comment has been minimized.

Copy link
Author

@csb0730 csb0730 commented Mar 9, 2020

@r10s, need to comment this:

btw. i am meanwhile also no longer convinced that the idea "delete all but N messages" is a good way to cleanup your device at all - even though i suggested that myself ;)

maybe sth. as deleting the largest messages first is better - deleting one video could make space for thousands of text messages. but also this is not that easy to implement and requires probably even more ui. and feature like that are in competition with tons of other things to do on 5 platforms - so, nothing for now :)

  1. Simply "delete all but N msgs" is NOT a good idea as a strict rule. We need at least starring of msgs to keep important ones.
  2. "delete all but N msgs" but msgs with big attachments first.
    I suspect that this will confuse many users (IMHO). This will generate more space quicker but will leave a disrupted history. Is that a transparent idea?

But want to metion some thoughts in next comment ...

@csb0730

This comment has been minimized.

Copy link
Author

@csb0730 csb0730 commented Mar 9, 2020

I mean we have two different situations at server and local device.

Server:
We have one folder for all msgs.
Msgs are sorted by design in cronological sequence.
Server doesn't know about chats!
==> Approach for purging:

  1. easy: a) delete oldest msgs by time or b) by max quantity of msgs.
  2. complicated: delete oldest msgs related to chats by a) or b)
    One of these two is done by #1206.

Local device:
We have several folders, one for every chat/group (what user sees).
Msgs are sorted in every chat in cronological sequence.
==> Approach for purging

  1. delete "oldest msgs" by time/age over all chats (!) #1309 ?
  2. delete "oldest msgs" by time/age per chat.
  3. delete "oldest msgs" keep last N over all chats (purges low runner by high runner chats!)
  4. delete "oldest msgs" keep last N per chat (original proposal in "limit retained history".

IMHO the two different technical structures might lead to different approaches for message purge at server and at local device.

Even two different use cases are existing:

  • Absolutely keep msgs by time/age. This leads to a strict disappearance of msgs, regardless of using device or not. To make this approach reliable and clear even purging at server and device is needed (sync).
  • Absolutely keep msgs by quantity. This only deletes msgs if new ones are coming in. IMHO it's not necessary to strictly sync with server. But this depends.

I think #1206 and #1309 are the basics for deleting msgs and could serve for N-limitation later.

@csb0730

This comment has been minimized.

Copy link
Author

@csb0730 csb0730 commented Mar 9, 2020

Last remark: Every approach should include a possibility to keep important messages.

@searoso

This comment has been minimized.

Copy link

@searoso searoso commented Mar 10, 2020

Just a suggestion. Is it possible to give user a choice: delete the messages or move them to kind of "DeltaChat-archive" folder?

Some people have inbox of 200 MB and what it purged fast, some do self-hosting and have 30GB for their inbox, and some in-between need to purge, but want to archive the messages locally just as other regular emails.

@testbird

This comment has been minimized.

Copy link

@testbird testbird commented Mar 16, 2020

maybe sth. as deleting the largest messages first is better - deleting one video could make space for thousands of text messages.

I'd say this might be handled simpler, by default (with much less confusion!)separately with a global gallery of pictures and documents that allows to delete attachments, ideally with the option to sort by size.
Only if the auto-delete can not meet the (soft) quota limits anymore, or the amount of to-be-deleted messages is very high, maybe point the user to the gallery to choose attachments to delete.

Every approach should include a possibility to keep important messages.

Just a suggestion. Is it possible to give user a choice: delete the messages or move them to kind of "DeltaChat-archive" folder?

All these, and more have gone into this design.
And my conclusion was that only the "keep last N per chat" can be synced between devices with a small set of reasonable, robust rules (given in the linked postings).

@hpk42

This comment has been minimized.

Copy link
Contributor

@hpk42 hpk42 commented Mar 17, 2020

Please @testbird -- i'd be grateful if you stop posting unasked-for advise on our trackers and support forum. We need more contributors who help and have a shitload of suggestions already -- and we had many discussions about our interactions with you before. Everyone involved is making their choices also with what kind of suggestions they feel able to, and want to engage. Nobody is forced to use Delta Chat or appreciate our way of going about things, there are many other good tools as well.
I hope you are fine and have a good sourrounding ... take care!

@searoso

This comment has been minimized.

Copy link

@searoso searoso commented Mar 17, 2020

@r10s

This comment has been minimized.

Copy link
Member

@r10s r10s commented Mar 17, 2020

with most users, the communication is no problem. and yes, many, many comments are helpful.

if you read issues here and in the forum, follow the mailinglist or irc, you will see that we do listen to the users and appreciate helpful and constructive feedback.

so, i do not think, we will make the issue tracker private just because of very view users.

@r10s

This comment has been minimized.

Copy link
Member

@r10s r10s commented Mar 17, 2020

Can we employ something to get value from this intentions? Maybe it's
possible to curate a table in feature proposal pinned post

there is https://support.delta.chat/c/features (the first post also has an curated overview).

what is actually going on is visible by the open prs and sometimes with open projects, but this is a bit hard to say - priorities shift, and if some volunteer wants do do sth. this may happens without any roadmap :)

@testbird

This comment has been minimized.

Copy link

@testbird testbird commented Mar 18, 2020

I really thought twice "should I send this."

Thank you. Such rules are always appropriate to remain respectful, and inappropriate when used to keep development under central control.

Interestingly, my first thought was also improving the issue management, actually going through, working out, sorting, consolidating, and tagging the issues. You may look at the list of the issues that I filed. Even though those of the original android and core seem to have gone, just as the github wiki pages.

In a good surrounding folks can work, collaboratively and most rewarding towards those fixes and options that would make deltachat work easily and well for a very diverse range of users. That's usually a feature of free software. It's different from working towards just one centrally, but generally non-fitting, interest.

@testbird

This comment has been minimized.

Copy link

@testbird testbird commented Mar 18, 2020

if some volunteer wants do do sth. this may happens without any roadmap :)

And without including nor considering prior work.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
5 participants
You can’t perform that action at this time.