Feature Vision Discussion: History

[sholderbach]

This issue is part of the larger vision discussion managed in #63 to brainstorm and prioritize ideas around features that would make reedline as a line editor for you both viable as well as pleasant to use. Feel free to up-vote features you depend on in other tools/shells as well as suggest new ideas!

History Features

• Ability to save commands in history
• Ability to save multi-line commands in history
• Ability to traverse backward and forward within history via keybindings
• Ability to search history
• Ability to search history and travese backward and forward within the search criteria
• Ability to have rich data within the history, such as:
  • run time in ms
  • date/time command was ran
  • sessionid of who ran the command assuming multiple sessions
• Ability to have multiple sessions successfull write to history file(s)
• Ability for each session to have it's own history or the appearance of it's own history so within history searching in a single session you only see the commands you created in that session
• Ability to send all commands from all sessions into a global history which search interfaces can use
• Ability to have plugable technologies to make history searching and recall extensible

[fdncred]

Just iterating on history a bit here.

An idea I had was to make reedline write a global history in a sqlite3 db (in WAL mode to handle concurrent reads & writes). Then we make each individual session write a local history that writes to reedline_history_sessionid.txt and then send that history item to sqlite. This way, you'd only see commands in your session when you use up/down but if you search history, it would search the sqlite db.

The benefit is that you solve the concurrency problem where you have 10 open tabs of shells using reedline at one time trying to write to the same single history.txt file.

Another option is to use the sqlite3 db, as described above, but also have a session id column in the table so every session would always update this singular sqlite db using it's own session id and never write a reedline_history_sessionid.txt file at all. Then make up/down just query out of that db based on it's sessionid and allow the search to query globally without the sessionid. This would probably simplify things but it moves all history to one sqlite db.

These sqlite options also tick some of the boxes above by having a rich history with sessionid, datetime, runtime perhaps, and other things that can be easily stored in a db.

[sholderbach]

Current voting

Vote for the topic(s) you care about by selecting the corresponding emoji. (No judgement based on the emojis sentiment!)

• 👍 Multiline support in history
• 👎 Metadata in history (last used time, wall time...)
• 😄 History search with navigation
• 🎉 History writing multiple sessions succesfully to file
• 😕 Database backed history to handle multiple sessions
• ❤️ Present local history
• 🚀 Make global history accessible
• 👀 History extensions/plugins

[fdncred]

As a way of being more verbose and transparent. Below are some of the ideas I've had with history. please add your own vision/ideas so we can collaborate and come up with something beyond cool.

The most important thing to me, other than having it function the way it should normally with up/down arrows, is that we have enough information to do fancy things in the future. To that end, let's list some fancy things we'd like to do to ensure we have enough information captured.

For reference this is that struct that I envisioned starting off with. I numbered them for easier reference.

#[derive(Debug, Clone, Ord, PartialOrd)]
pub struct HistoryItem {
      /// Primary Key, Unique Id
0.    pub history_id: Option<i64>,
      /// Entire command line
1.    pub command_line: String,
      /// Command part of the command line
2.    pub command: String,
      /// Parameters part of the command line
3.    pub command_params: Option<String>,
      /// Current working directory
4.    pub cwd: String,
      /// How long it took to run the command
5.    pub duration: i64,
      /// The exit status / return status of the command
6.    pub exit_status: i64,
      /// The pid of the running process
7.    pub session_id: i64,
      /// When the command was run
8.    pub timestamp: chrono::DateTime<Utc>,
      /// How many times was this command ran
9.    pub run_count: i64,
}

0. history_id: we have to have a primary key to have an index on. this item is for that. the only reason it's optional is because it should be an autonumber field. meaning that when you submit a record a new unique identifier is generated and you don't have to manually calculate the next historyid.
1. command_line, command, command_params: the next three i could give on changing to 1 item if we wanted to. the intent here is to use the nushell parser to extract these parts of a given pipeline. initially this would be mostly for statistics like we run the x command 90% of the time. and the most common use case for each is to do ls | each. the most commone sub commands for str are a, b, c. the most common parameters for ansi are blah. so, it's really for analysis to see how people are using nushell, assuming people would be willing to share their history.db with us. i realize we could parse it all out later but i figured, if the parser knows what the parts of the command line are already, why not just store it when we know it? i was also trying to future-proof the history.db so that if we think of a cool feature in the future, we'll have all the components already captured. this can also be used in conjunction with run_count for heuristics or AI type command prediction.
2. cwd: this is the current working directory, of course. i think it would be interesting to know which directories i run which commands in the most. so perhaps the blah/nushell folder cargo is ran the most. so, part of this is another statistics thing but also maybe a heuristics thing. because if i know what i use most often, perhaps i can predict what i'm going to use next. this can be using in conjunction with run_count to make our own internal z/zoxide clone where we rank the most popular folders used by run_count perhaps where a command was run from and we use that folder for the most popular suggestion on where to cd to.
3. duration: this is to understand how long commands take. of course it would be interesting for statistics and crunching the differences between each run of each and why they're longer or shorter. i think it could also be used for performance benchmarking to help understand historically if ls is getting slower or alternatively, what command has the worst performance or best performance. this could lead us into a direction of having more performant architectures.
4. exit_status: it's good to know what fails the most and why. we don't capture the why in my model here but status is a start. it would be interesting to know which command have non-zero exit statuses.
5. session_id: this is key to a global and local history functionality. i think global and local history can be captured in one file using the session id. the idea behind it is each terminal shell instance has it's own session id that's uniq, maybe a uuid is better. with a session id, we can make the searches work for only that session, however, if they want to search globally, we can ignore the session and just search the entire history. ideally we'd have some history_menu/tui for wading through this functionality and perhaps choosing wether you want to search locally or globally.
6. timestamp: i think it's good to capture when a command was run because you could probably do interesting things with it. obviously you can reconstruct command chronologically using a timestamp, but i'm sure there's other cool things too.
7. run_count: as mentioned earlier, this is really something to help with making a z/zoxide replacement specific to nushell/reedline. this would maybe have to be an sql update vs a sql insert in order to keep a running toll of commands. perhaps this type of information is in a separate table. speaking of tables, i'm fine with separating out tables to capture this information and more information. it doesn't have to be all in one place.

[stormasm]

As far as an implementation detail is concerned....

I think each one of the fields of the HistoryItem struct
should be stored in its own column as opposed to

/// This trait represents additional context to be added to a history (see [SqliteBackedHistory])
pub trait HistoryEntryContext: Serialize + DeserializeOwned + Default + Send {}

impl<T> HistoryEntryContext for T where T: Serialize + DeserializeOwned + Default + Send {}

/// A history that stores the values to an SQLite database.
/// In addition to storing the command, the history can store an additional arbitrary HistoryEntryContext,
/// to add information such as a timestamp, running directory, result...

I am not completely convinced of this though...
As I can see the benefits of the HistoryEntryContext too...

But it just seems simpler to me to understand...
As well as process, visualize, etc...

The downside is that if you want to add more stuff later...
Its more of an issue...
So I would be curious what other folks think as well...

[fdncred]

I think each one of the fields of the HistoryItem struct should be stored in its own column ...

I'm leaning this way too. I'd rather keep it as simple and approachable as possible.

[FilipAndersson245]

Having something simular to zoxide for your history would be amazing

[fdncred]

@FilipAndersson245 It's possible I think. The one thing missing is that we don't have how many times a command has been run and that's a key metric in zoxide's database and how they figure out which commands to run. We could put that information in the "more_info" column of the db, but someone would have to figure some things out.