Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rewrite #30

Merged
merged 23 commits into from Aug 1, 2021
Merged

Rewrite #30

merged 23 commits into from Aug 1, 2021

Conversation

vv9k
Copy link
Owner

@vv9k vv9k commented Jul 28, 2021

This PR is a big rewrite of the core functionality. Now the tags and entries are stored locally to a file in a OS cache directory. This allows for much better output and faster execution.

Closes: #6
Closes: #29

@lmburns
Copy link
Contributor

lmburns commented Aug 1, 2021

I'm unsure how to add a pull request to a pull request, but just a suggestion on adding color to the help message.

File: opt.rs

// --snip--

use clap::{Clap, AppSettings};
use wutag_core::Error;

pub const APP_NAME: &str = "wutag";
pub const APP_VERSION: &str = "0.4.0";
pub const APP_AUTHOR: &str = "Wojciech Kępka <wojciech@wkepka.dev>";
pub const APP_ABOUT: &str = "Tool to tag and manage tags of files.";

#[derive(Clap)]
#[clap(
    version = APP_VERSION,
    author = APP_AUTHOR,
    about = APP_ABOUT,
    global_setting = AppSettings::ColoredHelp,
    global_setting = AppSettings::ColorAuto,
)]

pub struct Opts {
    #[clap(short, long)]

// --snip--

And editing just to say that I like the new format, I'm unsure if the list command was there before, but I don't think it was in the same format.

I know that this is just a pull request and has not been merged to the master branch yet, but I'm also putting in here to say that when running

wutag rm '*' tag_name

I get the following errors:

/Users/lucasburns/test/lmformarks/formarks.plugin.zsh:
        ERROR tag `ftagged` doesn't exist - /Users/lucasburns/test/lmformarks/formarks.plugin.zsh

/Users/lucasburns/test/ftag:
        ERROR tag `ftagged` doesn't exist - /Users/lucasburns/test/ftag

But when listing files with that same tag using:

wutag list files
# or
wutag list tags

The files still appear.

Also, running

wutag clear '**'
# or
wutag clear <same_pattern_used_to_tag>

The files also still appear whenever listing.

Like I said, I know that this is pull request, but I wanted to let you know what errors I am getting. I'm in the process of learning Rust now and could possibly help some. I'd like to at least. My system is macOS if that matters at all.

@vv9k
Copy link
Owner Author

vv9k commented Aug 1, 2021

Thanks for checking out the PR! This is very much helpful :) I added the color to the help message - it's a worthy addition and fits well with the style of wutag.

I also updated how wutag rm and wutag clear works, it should now delete the tags from cache even when it fails to untag them on the file system. If you have any other issues please let me know. I will leave this PR open for now while I'll still polish it and cleanup the code.

@lmburns
Copy link
Contributor

lmburns commented Aug 1, 2021

The issue still isn't resolved and I get the same behavior when using rm or clear. I checked to make sure I was building the correct commit by checking the git log output.

I can run the following command many times in a row and nothing happens except for this output. I'm unsure if it is both deleting the xattrs and the path in the registry. Here's the output:

wutag rm '*.sh'

/Users/lucasburns/test/clap/count-tests.sh:

/Users/lucasburns/test/aaa.sh:

saving to /Users/lucasburns/Library/Caches/wutag.registry

It is different with this command (I think this is how it's supposed to be used). If the command is ran again, I get an error saying that the tag doesn't exists, but when listing the files and the tags (these are the only files with the tags) they still show up.

I'm unsure if you want the default behavior to remove all tags on files matching the glob, or if a tag has to be provided. If a tag is not provided, no error is thrown and it lists the files that are matching the rm command (as shown above).

wutag rm '*.sh' shell2

/Users/lucasburns/test/clap/count-tests.sh:
        X shell2
/Users/lucasburns/test/aaa.sh:
        X shell2
saving to /Users/lucasburns/Library/Caches/wutag.registry

After running above command:

$ wutag list files
/Users/lucasburns/test/clap/count-tests.sh
/Users/lucasburns/test/aaa.sh

$ wutag list tags
shell2

I think the behavior would be to delete the tag from the xattr and the registry if there are no files that contain the tags. I wonder if macOS has anything to do with it.


Showing errors if tag doesn't exist, and saying it is removing others.

wutag rm '*.d' jjj

/Users/lucasburns/projects/rust/repos_example/wutag/target/release/libwutag_core.d:
        ERROR tag `jjj` doesn't exist - /Users/lucasburns/projects/rust/repos_example/wutag/target
/release/libwutag_core.d

/Users/lucasburns/projects/rust/repos_example/wutag/target/release/wutag.d:
        ERROR tag `jjj` doesn't exist - /Users/lucasburns/projects/rust/repos_example/wutag/target
/release/wutag.d

/Users/lucasburns/projects/rust/repos_example/wutag/target/release/deps/anyhow-f4d68ea32d12b485.d:
        X jjj
/Users/lucasburns/projects/rust/repos_example/wutag/target/release/deps/walkdir-fdf07dfba53e1aa8.d
:
        X jjj

Using the following does work:

wutag edit <tag> --color 0x1f1f1f

Another issue I am having (though I am unsure if this is default behavior), is that whenever I list files or list tags, the entire system is traversed every time. And whenever I do:

wutag -d $PWD list files
# or
wutag -d . list files

the entire file system is also traversed.

If there is anything that you'd like me to try out to see if I can resolve the problem, I can. Also, all the tests are passed.

@lmburns
Copy link
Contributor

lmburns commented Aug 1, 2021

cargo run

This post should be much more complete and show every command.

set -- Works

$ cargo run -- set 'runner.rs' rust                                                         ─╯
    Finished dev [unoptimized + debuginfo] target(s) in 0.07s
     Running `/Users/lucasburns/projects/rust/repos_example/wutag/target/debug/wutag set runner.rs rust`
/Users/lucasburns/projects/rust/repos_example/wutag/src/runner.rs:
        + rust
saving to /Users/lucasburns/Library/Caches/wutag.registry

edit -- Works

  • However, no error is thrown if there is an invalid tag given.
cargo run -- edit rust --color 0xFFFFFF                                                    ─╯
    Finished dev [unoptimized + debuginfo] target(s) in 0.06s
     Running `/Users/lucasburns/projects/rust/repos_example/wutag/target/debug/wutag edit rust --color 0xFFFFFF`
rust ==> rust
saving to /Users/lucasburns/Library/Caches/wutag.registry

cp -- Displaying that it works, but doesn't copy

$ cargo run -- cp runner.rs config.rs
    Finished dev [unoptimized + debuginfo] target(s) in 0.06s
     Running `/Users/lucasburns/projects/rust/repos_example/wutag/target/debug/wutag cp runner.rs config.rs`
/Users/lucasburns/projects/rust/repos_example/wutag/src/config.rs:
saving to /Users/lucasburns/Library/Caches/wutag.registry

However, listing files afterward does not display that file as being tagged:

$ cargo run -- list files
    Finished dev [unoptimized + debuginfo] target(s) in 0.07s
     Running `/Users/lucasburns/projects/rust/repos_example/wutag/target/debug/wutag list files`
/Users/lucasburns/projects/rust/repos_example/wutag/src/runner.rs
/Users/lucasburns/test/aaa.sh
/Users/lucasburns/test/clap/count-tests.sh

clear -- Backtrace

$ cargo run -- clear '*.rs'
Finished dev [unoptimized + debuginfo] target(s) in 0.09s
     Running `/Users/lucasburns/projects/rust/repos_example/wutag/target/debug/wutag clear '*.rs'`thread 'main' panicked at 'removal index (is 1627790360219654000) should be < len (is 1)', library/alloc/src/vec/mod.rs:1355:13
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

rm -- Displays that it works, but doesn't

If the tag is set, when removing the tag the first time, no backtrace is given; however, if the command is ran again, a backtrace is given as shown below. This should not display the rust tag at all because it should've been deleted from the xattrs the first time.

Also, I'm not sure if the other errors are supposed to be shown or if it is just supposed to show the files that it is supposed to remove the tags from.

cargo run -- rm '*.rs' rust                                                               ─╯
    Finished dev [unoptimized + debuginfo] target(s) in 0.06s
     Running `/Users/lucasburns/projects/rust/repos_example/wutag/target/debug/wutag rm '*.rs' rust`
/Users/lucasburns/projects/rust/repos_example/wutag/src/opt.rs:
        ERROR tag `rust` doesn't exist - /Users/lucasburns/projects/rust/repos_example/wutag/src/opt.rs

/Users/lucasburns/projects/rust/repos_example/wutag/src/config.rs:
        ERROR tag `rust` doesn't exist - /Users/lucasburns/projects/rust/repos_example/wutag/src/config.rs

/Users/lucasburns/projects/rust/repos_example/wutag/src/runner.rs:
thread 'main' panicked at 'called `Option::unwrap()` on a `None` value', src/registry.rs:90:32
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
        X rust

After running the above command twice and getting that error message, when listing the files again, the path is still displayed:

$ cargo run -- list files                                                                   ─╯
    Finished dev [unoptimized + debuginfo] target(s) in 0.22s
     Running `/Users/lucasburns/projects/rust/repos_example/wutag/target/debug/wutag list files`
/Users/lucasburns/test/clap/count-tests.sh
/Users/lucasburns/projects/rust/repos_example/wutag/src/runner.rs
/Users/lucasburns/test/aaa.sh

search -- This displays what the problem is

If a tag is set in the directory and I search for it, it is shown:

$ cargo run -- search 'rust'                                                                ─╯
    Finished dev [unoptimized + debuginfo] target(s) in 0.06s
     Running `/Users/lucasburns/projects/rust/repos_example/wutag/target/debug/wutag search rust`
/Users/lucasburns/projects/rust/repos_example/wutag/src/runner.rs: rust

If I remove the tag:

$ cargo run -- rm '*.rs' rust                                                               ─╯
    Finished dev [unoptimized + debuginfo] target(s) in 0.18s
     Running `/Users/lucasburns/projects/rust/repos_example/wutag/target/debug/wutag rm '*.rs' rust`
/Users/lucasburns/projects/rust/repos_example/wutag/src/opt.rs:
        ERROR tag `rust` doesn't exist - /Users/lucasburns/projects/rust/repos_example/wutag/src/opt.rs

/Users/lucasburns/projects/rust/repos_example/wutag/src/util.rs:
        ERROR tag `rust` doesn't exist - /Users/lucasburns/projects/rust/repos_example/wutag/src/util.rs

/Users/lucasburns/projects/rust/repos_example/wutag/src/registry.rs:
        ERROR tag `rust` doesn't exist - /Users/lucasburns/projects/rust/repos_example/wutag/src/registry.rs

/Users/lucasburns/projects/rust/repos_example/wutag/src/config.rs:
        ERROR tag `rust` doesn't exist - /Users/lucasburns/projects/rust/repos_example/wutag/src/config.rs

/Users/lucasburns/projects/rust/repos_example/wutag/src/runner.rs:
        X rust
/Users/lucasburns/projects/rust/repos_example/wutag/src/main.rs:
        ERROR tag `rust` doesn't exist - /Users/lucasburns/projects/rust/repos_example/wutag/src/main.rs

saving to /Users/lucasburns/Library/Caches/wutag.registry

And search again, it is gone:

$ cargo run -- search 'rust'                                                                ─╯
    Finished dev [unoptimized + debuginfo] target(s) in 0.06s
     Running `/Users/lucasburns/projects/rust/repos_example/wutag/target/debug/wutag search rust`

However, if I list the files, it is still there:

$ cargo run -- list files                                                                   ─╯
    Finished dev [unoptimized + debuginfo] target(s) in 0.06s
     Running `/Users/lucasburns/projects/rust/repos_example/wutag/target/debug/wutag list files`
/Users/lucasburns/projects/rust/repos_example/wutag/src/runner.rs
/Users/lucasburns/test/clap/count-tests.sh
/Users/lucasburns/test/aaa.sh

-d

Doesn't affect which files are listed.

$ cargo run -- -d . list files                                                              ─╯
    Finished dev [unoptimized + debuginfo] target(s) in 0.28s
     Running `/Users/lucasburns/projects/rust/repos_example/wutag/target/debug/wutag -d . list files`
/Users/lucasburns/test/aaa.sh
/Users/lucasburns/test/clap/count-tests.sh
/Users/lucasburns/projects/rust/repos_example/wutag/src/runner.rs

-m

I tested max-depth earlier when setting tags and it worked.

@vv9k
Copy link
Owner Author

vv9k commented Aug 1, 2021

The issue still isn't resolved and I get the same behavior when using rm or clear. I checked to make sure I was building the correct commit by checking the git log output.

I missed a few bugs, should be better now.

I can run the following command many times in a row and nothing happens except for this output. I'm unsure if it is both deleting the xattrs and the path in the registry. Here's the output:

wutag rm '*.sh'

/Users/lucasburns/test/clap/count-tests.sh:

/Users/lucasburns/test/aaa.sh:

saving to /Users/lucasburns/Library/Caches/wutag.registry

This should not display the paths anymore unless the tags are actually deleted.

Another issue I am having (though I am unsure if this is default behavior), is that whenever I list files or list tags, the entire system is traversed every time. And whenever I do:

wutag -d $PWD list files
# or
wutag -d . list files

the entire file system is also traversed.

If there is anything that you'd like me to try out to see if I can resolve the problem, I can. Also, all the tests are passed.

This shouldn't be the case, current implementation doesn't touch the file system except for the cache file from which the list of entries or tags is read. The -d flag only applies to commands that take a pattern as an input (it should be noted in the help message).

I saw you also got a panic when doing a clear, that should also be resolved now.

@vv9k
Copy link
Owner Author

vv9k commented Aug 1, 2021

cargo run

This post should be much more complete and show every command.
...

Please try the latest commit 3c5dea7 , it should fix most of the issues.

@vv9k
Copy link
Owner Author

vv9k commented Aug 1, 2021

I also noticed that wutag clear doesn't work at all, it was because it was only executed when the --verbose flag was passed. Should work in all cases now. Tested locally.

@vv9k
Copy link
Owner Author

vv9k commented Aug 1, 2021

A sidenote, testing out some of the version of the code I ended up with files without tags that are in cache. To clear it up I had to manually remove $XDG_CACHE_HOME/wutag.registry. I think I'll add a subcommand that lets the user clear the cache easily.

@lmburns
Copy link
Contributor

lmburns commented Aug 1, 2021

I removed the registry manually, and when running both clear and rm this time, they work. However, clear does not display anything showing that it cleared the tags, rm does though. The only issue that is still there is that when listing the files and tags both, the file that was just either cleared or rm'd is still being shown. Would this have anything to do with the way the registry is being stored across systems?

Screen Shot 2021-07-31 at 11 34 44 PM

Would it also be possible to have an option to maybe implement this function list_entries_with_tags from the command line to show all files and tags beside each other without having to specifically search for the tags to get that display?

Perhaps, wutag list files-tags, or files-n-tags

@vv9k
Copy link
Owner Author

vv9k commented Aug 1, 2021

I removed the registry manually, and when running both clear and rm this time, they work. However, clear does not display anything showing that it cleared the tags, rm does though. The only issue that is still there is that when listing the files and tags both, the file that was just either cleared or rm'd is still being shown. Would this have anything to do with the way the registry is being stored across systems?

I don't think the way it's stored matters in this case. Clear doesn't display anything by default, only with --verbose or -v. My initial thought was that when clearing usually a lot of files are cleared so there would be a lot of output but I think it's better to keep it the same across all subcommands so I removed the --verbose flag.

Screen Shot 2021-07-31 at 11 34 44 PM

Would it also be possible to have an option to maybe implement this function list_entries_with_tags from the command line to show all files and tags beside each other without having to specifically search for the tags to get that display?

Perhaps, wutag list files-tags, or files-n-tags

I added a flag so that you can now do wutag list files --with-tags or wutag list files -t and it will display:

❯ cargo r -- list files --with-tags
    Finished dev [unoptimized + debuginfo] target(s) in 0.02s
     Running `target/debug/wutag list files --with-tags`
/home/wojtek/dev/rust/wutag/target/CACHEDIR.TAG: trash ...
/home/wojtek/dev/rust/wutag/src/opt.rs: test123
/home/wojtek/dev/rust/wutag/src/config.rs: test123
/home/wojtek/dev/rust/wutag/target/debug: trash ...
/home/wojtek/dev/rust/wutag/target/.rustc_info.json: trash ...
/home/wojtek/dev/rust/wutag/src/util.rs: test123
/home/wojtek/dev/rust/wutag/src/main.rs: test123
/home/wojtek/dev/rust/wutag/src/runner.rs: test123
/home/wojtek/dev/rust/wutag/src/registry.rs: test123

@vv9k
Copy link
Owner Author

vv9k commented Aug 1, 2021

The only issue that is still there is that when listing the files and tags both, the file that was just either cleared or rm'd is still being shown. Would this have anything to do with the way the registry is being stored across systems?

Would you mind removing the registry again and noting exactly what commands you issue to get into this state? I cannot really reproduce it.

EDIT:

I was able to find the issue, it should be working now

@vv9k
Copy link
Owner Author

vv9k commented Aug 1, 2021

I think I will merge this PR for now as it contains way too many changes. If you find any new bugs please report them in separate issues :) Thanks!

@vv9k vv9k changed the title Add cached state Rewrite Aug 1, 2021
@vv9k vv9k merged commit 60121a6 into master Aug 1, 2021
@lmburns
Copy link
Contributor

lmburns commented Aug 1, 2021

Awesome, it is working now! The tags and files both are no longer being displayed after removing the tags or clearing the tags. I like how it shows that it is cleared as well. I have been using jdberry's tag for macOS and wrote a fairly long script for it that also included wutag, but I think that I may start using wutag only

@vv9k vv9k mentioned this pull request Aug 5, 2021
@vv9k vv9k deleted the state branch November 8, 2021 16:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Keeping track of all tags for a global search Add some sort of persistence
2 participants