Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Deleting history #592

Closed
ellie opened this issue Nov 2, 2022 · 15 comments · Fixed by #791
Closed

Deleting history #592

ellie opened this issue Nov 2, 2022 · 15 comments · Fixed by #791

Comments

@ellie
Copy link
Member

ellie commented Nov 2, 2022

So this has been proposed for a super long time.

It's taken a while, partly because deletion with sync is difficult. Deleting on one machine would result in it coming back the next time a sync happens, so we need to ensure deletion across all machines

I'm starting a new issue to track this, as it's actively being worked on

I wrote up my plan in a comment on #70, but copying it here for clarity


tl;dr for the plan, which I'd like to break down a bit more and HOPEFULLY get in for a v12 release. Or at least partly!

Deleting things, finally, as everyone has been asking me for it

(cue brain dump)

1. Switch to event log sync

I started on this here: #390

The difficulty here is that actually deleting the entire row will change the count of rows. Syncing two append-only logs is much much easier and simpler than syncing two arbitrary length sets of data, especially when each installation of atuin is both read and write (so technically multi-master).

The premise with this solution is that instead of syncing up history items, as we do at the moment, we sync up events. I'm proposing two kinds of event. A CREATE event, and a DELETE event. For backwards compatibility, we'd want to add a CREATE event for every currently-existing item of history.

To delete an item of history, we push up a DELETE event to the log. This is still only an append, so we can retain our simple sync logic + keep things nice and easy.

If we replay the log on a fresh client, the history item will end up being created + then deleted. Mostly there, and mostly achieves the use case of "I don't want this thing showing up in my history".

Once that works, we could expand it to also blank out the data section of a CREATE event when a DELETE event has been created, though that would cause limitations in future sync methods (I was conisdering something based on merkle trees).

2. Implement UI for deletion

I'd like this to be smooth, obvious, and not some arbitrary key combination. But also not easy to do by accident! We may also want the option to delete a single occurence of a command, but not delete all historical occurances of it (and vice-versa).

To allow for this, it would be good to be able to open a "context menu" on a single item of history. I'd suggest using the tab character for this.

Scroll back to the item you wish to delete. Hit tab, and we switch to the menu item for that history. As well as showing extra information about it (directory executed in, some other context, etc), we can have an option for deleting it!

Might want some kind of UI hint to make it obvious that tab does stuff

3. done...?

Probably not, I imagine some things will break. I'd like to roll this out gradually, as we need to maintain backwards compatibility. Will probably save old style history rows and event rows side-by-side, and start by just syncing CREATE events up and not even implementing DELETE until the next version.

@ellie ellie pinned this issue Nov 2, 2022
@ellie
Copy link
Member Author

ellie commented Nov 4, 2022

Just merged the first part of this!

#390

You should see a new table created locally + tracking events

@tobiasvl
Copy link

I'd like this to be smooth, obvious, and not some arbitrary key combination. But also not easy to do by accident!

I'd personally love it if I could just highlight the entry and simply press Del to delete it, or perhaps Shift+Del like Chrome does it in the URL bar. This seems smooth and obvious and non-arbitrary to me, and especially the latter one is hard to do by accident.

@eripa
Copy link
Contributor

eripa commented Jan 29, 2023

In addition to selectively deleting records it would be good to have a way to configure sort of retention or max age/limit. I.e. "remove any history items older than 5y or 500k lines" or similar.

@ellie
Copy link
Member Author

ellie commented Jan 29, 2023

In addition to selectively deleting records it would be good to have a way to configure sort of retention or max age/limit. I.e. "remove any history items older than 5y or 500k lines" or similar.

For sure! I've been thinking about that too. It has the same problem of breaking sync though 🙃

@manfredlotz
Copy link

From a usage point of view I think the UI part for deleting is not so important.

When wanting to delete something in history I

  • would like to check what to delete by running atuin search <srchpat1> ...
  • and then deleting those entries by running atuin delete <srchpat1> ...

PS: For me the ability to delete items in the history is also very important.

@mentalisttraceur
Copy link
Contributor

You might find it helpful to

  1. look at what people have figured out in shared text editing (whether multi-user or single user racing with async editor plugins), because they have to deal with what is fundamentally the same problem; and

  2. also look at how undo is implemented, because a deletion is in many ways like an undo.

I remember reading some blog posts about implementing the Xi editor a few years ago were fairly enlightening for me, which might be relevant here. In particular this one about CRDT and async undo.

@ellie
Copy link
Member Author

ellie commented Feb 26, 2023

From a usage point of view I think the UI part for deleting is not so important.

When wanting to delete something in history I

* would like to check what to delete by running `atuin search <srchpat1> ...`

* and then deleting those entries by running `atuin delete <srchpat1> ...`

PS: For me the ability to delete items in the history is also very important.

Super true. I'll probably release this first and think about the UI later

@ellie
Copy link
Member Author

ellie commented Feb 26, 2023

You might find it helpful to

1. look at what people have figured out in shared text editing (whether multi-user or single user racing with async editor plugins), because they have to deal with what is fundamentally the same problem; and

2. also look at how _undo_ is implemented, because a deletion is in many ways like an undo.

I remember reading some blog posts about implementing the Xi editor a few years ago were fairly enlightening for me, which might be relevant here. In particular this one about CRDT and async undo.

I did spend a while reading this literature + thinking about it, but they have a much harder problem to solve - that of multiple users + edit conflicts.

Neither of those problems really apply here, we just want to append stuff - and that can either be a CREATE or a DELETE

@mentalisttraceur
Copy link
Contributor

mentalisttraceur commented Feb 28, 2023

@ellie Cool, sorry it ended up not relevant. Been a few years since I read those.

Was just throwing it out there in case those more general solutions (or the libraries that implement them) for distributed edits ended up being a good fit.

Because to me a synced create+delete log felt like the beginnings or special case of the same problem shape, and in your idea of then going back to erase the data in the create entry I saw some (very superficial/abstract) parallels to how the Xi editor design docs mentioned leaving tombstone entries for deleted text.

@mentalisttraceur
Copy link
Contributor

mentalisttraceur commented Feb 28, 2023

I also think it's relevant because I feel like Atuin's success (and forecast of more success) is big enough that desire (or even need, maybe) for more complex history editing is just around the corner.

For example, the Xi blog post talks a bit about undo groups, and I wouldn't be surprised if eventually Atuin's community grows plugins that add functionality on top of Atuin, and then that leads to things like "would be great if I could sync a transaction of history edits, so that the first edit only goes through if another edit also goes through", and so on.

Of course we should not let that get in the way of building something that meets current needs now.

But if you like the idea then perhaps it doesn't need to get in the way - perhaps it ends up being similarly easy to implement (with so much literature to guide it), or perhaps we can end up finding a good reusable library out of the core of some multi-user/async editor.

Not trying to talk you into something you don't want to do though! If you already have vision/inertia/motivation for the simple create+delete log approach, or the CRDT/etc approach doesn't seem like the right move at least at this time, 👍

@ellie
Copy link
Member Author

ellie commented Mar 8, 2023

I saw some (very superficial/abstract) parallels to how the Xi editor design docs mentioned leaving tombstone entries for deleted text.

For sure! It actually has a lot of similarities with some other data problems too - eg the clickhouse mergetree.

I feel like Atuin's success (and forecast of more success)

Haha thank you for the positive energy 🚀

Not trying to talk you into something you don't want to do though! If you already have vision/inertia/motivation for the simple create+delete log approach, or the CRDT/etc approach doesn't seem like the right move at least at this time, 👍

So the idea here is that something like a CRDT would exist one layer of abstraction above this. With any CRDT, you need a mechanism for actually syncing the communication/data behind it. The event sync would be that - there's nothing stopping anyone in the future from implementing a CRDT on top of this. The events as currently written are basically just the same as the history blob sync, except they're just tagged with a type. The event sync doesn't care if it's a create, a delete, undo, or some other future event type - it's only when we actually process the events that this matters, and having something like a CRDT may be useful in the future!

I'm hoping with this approach we can solve our current problems, but still allow for future flexibility.

@hlascelles
Copy link

Thanks for this @ellie! I can use delete with atuin search "foo" --delete... But is there a keybinding to delete a row when looking at the main atuin UI?

I may be searching for something, and see an out of date or sensitive line, and just want to highlight it and some key to remove it from history. Has that been discussed? I can open a new issue if needed. Thanks!

@bvergnaud
Copy link
Contributor

Hey, not Ellie here. 😁

In the current version of atuin, in the main UI you can hit Ctrl+o to inspect the highlighted entry, and in that new view, you can hit Ctrl+d to delete it. Hope that helps. 😉

@hlascelles
Copy link

Thank you @bvergnaud, that's the one! 🙇‍♂️

Neither CTRL+O keybinding or the CTRL+D are mentioned in the keybindings documentation: https://docs.atuin.sh/configuration/key-binding/#atuin-ui-shortcuts

Searching for "delete" in there doesn't reveal it either.

Should I do a PR for all that in https://github.com/atuinsh/docs ?

@bvergnaud
Copy link
Contributor

Thank you @bvergnaud, that's the one! 🙇‍♂️

Neither CTRL+O keybinding or the CTRL+D are mentioned in the keybindings documentation: https://docs.atuin.sh/configuration/key-binding/#atuin-ui-shortcuts

Searching for "delete" in there doesn't reveal it either.

It's a pretty recent feature, it was added in v18 in early february. I guess the documentation was forgotten.

Should I do a PR for all that in https://github.com/atuinsh/docs ?

I don't want to speak for the maintainers here but if you've got the time for it, I'm sure they'll appreciate it. 🙂

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

7 participants