command expansions

[QiBaobin]

#3134

[QiBaobin]

only support filename val for now:
bc %val{filename}

support simple sh command like:

open %sh{fd\ 16_theme}

[sbromberger]

Just curious: does this quote the filename so that if you pass it to sh it doesn't fall victim to the IFS boundary?

[QiBaobin]

Just curious: does this quote the filename so that if you pass it to sh it doesn't fall victim to the IFS boundary?

I think we can use sh echo "%val{filename}"

[trink]

FYI: This doesn't handle substitution in the keymap arguments.

[keys.normal]
"C-s" = [":sh hx_rcs_write.sh '%val{filename}'", ":w"] # version control the previous and write out the new

[QiBaobin]

"C-s" = [":sh hx_rcs_write.sh '%val{filename}'", ":w"] 

@trink , it shall work now, please take anther look, thanks!

[QiBaobin]

support %val {filename} %val {dirname} and %sh {cmd} now, we can also nest them like %sh { echo %val {dirname} }, also addressed the 'have to escape space' issue as @the-mikedavis suggested.

[pickfire]

Shouldn't this return an error too?

[pickfire]

Shouldn't we iterate over char directly instead? I think unicode might crash in this case when we are reading bytes within multiple bytes char.

Also, the count with unicode characters is incorrect now, as_bytes may increment i more times than the actual character length str.len.

[QiBaobin]

the b is a byte so that this won't crash.As we only use ascii chars in our patterns below, I didn't treat them as Unicode char, this shall work until we introduce other Unicode pattern beside %val etc.

[g-re-g]

If these patterns are coming from user input there's always a chance that they can be multi-byte, or even multi code-point "characters". Because of that you might want the parser to be aware / tolerant of it.

If that is the case then String.chars probably still isn't enough and you'll need something more unicode aware if you're interested in knowing a particular characters index as a human would see it. See this note from the rust std library:

let y = "y̆";
let mut chars = y.chars();
assert_eq!(Some('y'), chars.next()); // not 'y̆'

The typical crate I've used / seen used is https://crates.io/crates/unicode-segmentation . Specifically for this implementation you could use grapheme_indices https://unicode-rs.github.io/unicode-segmentation/unicode_segmentation/trait.UnicodeSegmentation.html#tymethod.grapheme_indices .

[trink]

The returned index will always be in bounds (the enumerator and len are both byte based). Trying something like the following doesn't cause any issues
sh echo %val{𒀀𒀀𒀀𒀀𒀀𒀀𒀀} // 4 byte char f0928080

I think this should remain as is.

However, downstream I am seeing something off in shellwords (some multi-byte characters are lost in the generated argument list, I am investigating it now but it is unrelated to this PR)
edit: #5738

[pickfire]

Should we auto-complete for users? I think %sh{echo hello shouldn't work.

[QiBaobin]

yes, above command won't expand and will be treated literately . The auto-complete is a very nice to have feature, how about we adding it after this PR.

[trink]

The added space between rep and next is a problem when you want to compose a single argument with the substitution. e.g.
:sh echo %val{filename},v
This currently produces two arguments "filename", ",v" instead of the desired "filename,v"

[QiBaobin]

Good point, will fix it once I have time.

[trink]

The returned index will always be in bounds (the enumerator and len are both byte based). Trying something like the following doesn't cause any issues
sh echo %val{𒀀𒀀𒀀𒀀𒀀𒀀𒀀} // 4 byte char f0928080

I think this should remain as is.

However, downstream I am seeing something off in shellwords (some multi-byte characters are lost in the generated argument list, I am investigating it now but it is unrelated to this PR)
edit: #5738

[danillos]

Thanks, @QiBaobin I really liked this feature, with that I could build:

F8 = ':open %sh { ~/.dotfiles/support/command.sh switch_to_spec %val{filename} }'
F10 = ':sh ~/.dotfiles/support/command.sh run_current_file_spec %val{filename}'

These scripts that I created are not perfect because I had to hard-code how to get the project dirname.

Could we have something like %val{project_dirname} expand to the absolute folder path used in hx . ?

Edit: Another useful variable is something like %val{line_number}, to use in commands like open it in github.

[ksdrar]

Hey guys, I got the nested commands working with %key{body}

You can try the changes in the branch cmd-expansions of my fork.

The output of the tests:

❯ cargo run --release
   Compiling pattern_matching v0.1.0 (/home/jesus/Development/rust/pattern_matching)
    Finished release [optimized] target(s) in 0.54s
     Running `target/release/pattern_matching`
[TEST] %val {filedir} => filedir
[PASSED 821.124µs] filedir == filedir

[TEST] %val {filedir%val {aasdfasdf}} => filediraasdfasdf
[PASSED 738.772µs] filediraasdfasdf == filediraasdfasdf

[TEST] %sh {rm %val{filedir},%val{filename}} => rm filedir,filename
[PASSED 647.805µs] rm filedir,filename == rm filedir,filename

[TEST] %sh {git blame %val{filename}:%val{linenumber}} => git blame filename:linenumber
[PASSED 666.177µs] git blame filename:linenumber == git blame filename:linenumber

[TEST] :sh (%sh {cat %val{filename} | grep rust}) > %val{filedir}/test.txt => :sh (cat filename | grep rust) > filedir/test.txt
[PASSED 660.74µs] :sh (cat filename | grep rust) > filedir/test.txt == :sh (cat filename | grep rust) > filedir/test.txt

[TEST] :sh bash -c (%sh {(%sh {cat %val{filedir}/test.txt}) > %val{filedir}/../%val{filename}_test.txt}) => :sh bash -c ((cat filedir/test.txt) > filedir/../filename_test.txt)
[PASSED 736.72µs] :sh bash -c ((cat filedir/test.txt) > filedir/../filename_test.txt) == :sh bash -c ((cat filedir/test.txt) > filedir/../filename_test.txt)

The bare-bones code:

fn test(to_test: &str, expected: &str) {
    let start = std::time::Instant::now();
    println!("\x1b[33m[TEST]\x1b[0m {to_test} => {expected}");
    let result = find_and_replace(to_test).unwrap_or_default();
    assert_eq!(result, expected);
    println!(
        "\x1b[32m[PASSED {:2?}]\x1b[0m {result} == {expected}\n",
        start.elapsed()
    );
}

fn main() {
    test("%val {filedir}", "filedir");

    test("%val {filedir%val {aasdfasdf}}", "filediraasdfasdf");

    test(
        "%sh {rm %val{filedir},%val{filename}}",
        "rm filedir,filename",
    );

    test(
        "%sh {git blame %val{filename}:%val{linenumber}}",
        "git blame filename:linenumber",
    );

    test(
        ":sh (%sh {cat %val{filename} | grep rust}) > %val{filedir}/test.txt",
        ":sh (cat filename | grep rust) > filedir/test.txt",
    );

    test(
        ":sh bash -c (%sh {(%sh {cat %val{filedir}/test.txt}) > %val{filedir}/../%val{filename}_test.txt})",
        ":sh bash -c ((cat filedir/test.txt) > filedir/../filename_test.txt)",
    );
}

fn find_and_replace(input: &str) -> anyhow::Result<String> {
    let regexp = regex::Regex::new(r"%(\w+)\s*\{([^{}]*(\{[^{}]*\}[^{}]*)*)\}").unwrap();

    replace_all(&regexp, input, |captures| {
        let keyword = captures.get(1).unwrap().as_str();
        let body = captures.get(2).unwrap().as_str();

        match keyword {
            "val" => Ok(String::from(body)),
            "sh" => Ok(String::from(body)),
            _ => anyhow::bail!("Unknown keyword {keyword}"),
        }
    })
}

fn replace_all(
    regex: &regex::Regex,
    text: &str,
    matcher: impl Fn(&regex::Captures) -> anyhow::Result<String>,
) -> anyhow::Result<String> {
    let mut it = regex.captures_iter(text).peekable();

    if it.peek().is_none() {
        return Ok(String::from(text));
    }

    let mut new = String::with_capacity(text.len());
    let mut last_match = 0;

    for cap in it {
        let m = cap.get(0).unwrap();
        new.push_str(&text[last_match..m.start()]);

        let replace = matcher(&cap)?;

        new.push_str(&replace);

        last_match = m.end();
    }

    new.push_str(&text[last_match..]);

    replace_all(regex, &new, matcher)
}

[danillos]

I’ve tested your branch, @ksdrar, and it worked perfectly. In my opinion, you can submit a Pull Request to replace the current one. I believe @QiBaobin will close this current PR once yours is opened.

Nice work.

[danillos]

@ksdrar I found an issue. I was migrating all keybindings that I did using my version to your version and this one didn't work.

# before
s = ':open %sh { ~/.dotfiles/helix/command.rb switch_to_spec %val{projectdir} %val{relative_filename}}'

# after
s = ':open %sh {~/.dotfiles/helix/command.rb switch_to_spec %sh{pwd} %val{filename}}'

It is not expanding %sh{pwd} and %val{filename}, If I understood, it first expands %sh

[danillos]

I think it should expand the most inner expressions first, and move to the next one only when there is nothing left to expand. For example:

filename = test.rs
line_number = 123

Command:
%sh { add_suffix.sh %sh { add_line_number.sh %sh { add_prefix.sh %val{filename} } } }

Expansion Steps:

0. %sh { add_suffix.sh %sh { add_line_number.sh %sh { add_prefix.sh %val{filename} } } }
1. %sh { add_suffix.sh %sh { add_line_number.sh %sh { add_prefix.sh test.rs } } }
2. %sh { add_suffix.sh %sh { add_line_number.sh prefix_test.rs } }
3. %sh { add_suffix.sh prefix_test.rs:%val{line_number} }
4. %sh { add_suffix.sh prefix_test.rs:123 }
5. prefix_test_suffix.rs:123

Scripts:

add_prefix.sh: adds `prefix_{string}` to a string
add_suffix.sh: adds `{string}_suffix` to a string
add_line_number.sh: adds `{string}:%val{line_number}` to a string

[ksdrar]

@danillos Can you check if it's working now with the last commit?

[danillos]

@danillos Can you check if it's working now with the last commit?

Yes, this example is now working.

s = ':open %sh {~/.dotfiles/helix/command.rb switch_to_spec %sh{pwd} %val{filename}}'

[danillos]

@ksdrar could you open your changes as a new PR for replace this one? I could do that too.