Allow regex search #9228

PankajBhojwani · 2021-02-20T00:22:04Z

Summary of the Pull Request

Initial stage of implementing regex search

Todos left:

Allow find previous
Wraparound when search fails
Change the icon in the search box

References

#3920

PR Checklist

Closes #xxx
CLA signed. If not, go over here and sign the CLA
Tests added/passed
Documentation updated. If checked, please file a pull request on our docs repo and link it here: #xxx
Schema updated.
I work here

Validation Steps Performed

PankajBhojwani · 2021-02-24T00:48:31Z

src/buffer/out/search.cpp

+            // Regex iterators are not bidirectional, so we cannot create a reverse iterator out of one
+            // So, if we want to find the previous match, we need to manually progress the iterator
+            while (matches_begin != matches_end)
+            {
+                desired = matches_begin;
+                ++matches_begin;
+            }


There really should be a better way to do this

KalleOlaviNiemitalo · 2021-02-24T13:58:52Z

src/buffer/out/search.cpp

+
+    // Create the regex object and iterator
+    std::wregex regexObj{ _inputString };
+    auto matches_begin = std::wsregex_iterator(concatAll.begin(), concatAll.end(), regexObj);


If the user accidentally types a pathological regex and the search takes too long, is there any way to abort it? It doesn't look like std::regex_iterator supports any kind of cancellation token.

I wonder if cancellation could be implemented by throwing an exception from operators of a custom iterator type. Would there be a risk that such an exception can skip some cleanup in the regexp algorithm, or that the algorithm can copy part of the input to an internal buffer and then spend excessive time examining that buffer, without using the original iterators?

src/buffer/out/search.cpp

src/cascadia/TerminalControl/SearchBoxControl.cpp

PankajBhojwani · 2021-02-26T00:02:52Z

#8588 makes our searching much more efficient, so we will wait till that merges before merging this one.

github-actions · 2021-10-26T18:06:55Z

@check-spelling-bot Report

Unrecognized words, please review:

AAAAA
AAAAAAAAAAAAA
BBBBB
BBBBBBBB
Gravell
It'd
ok'd
pws
qos
RIS
sxs
sys
We'd
ZZZZZ

Previously acknowledged words that are now absent

aef apos aspnet bc boostorg BSODs Cac ci COINIT conpixels cplinfo crt cw cx dahall DEFAPP DEFCON df dh dw eb EK ev exeuwp fde fea fmtlib gb gh Gravell's hc hh hk hotkeys HPCON hu isocpp Kd KF KJ KU KX lk llvm mintty msvcrtd Nc NVDA Nx oa OI Oj Oo oq Ou Ov pb pinam pv pw px py qi QJ qo QOL remoting ri Rl rmdir ru smalllogo splashscreen storelogo sx sy sz td tf tl tt uk ul Unk unte vcrt vf vk wx xa xamarin xy Yw yx YZ Zc zd zh ZM zu zy

Some files were were automatically ignored

These sample patterns would exclude them:

^src/host/ft_uia/run\.bat$
^src/host/runft\.bat$
^src/host/runut\.bat$
^src/terminal/adapter/ut_adapter/run\.bat$
^src/terminal/parser/delfuzzpayload\.bat$
^src/terminal/parser/ft_fuzzer/run\.bat$
^src/terminal/parser/ft_fuzzwrapper/run\.bat$
^src/terminal/parser/ut_parser/run\.bat$
^src/tools/integrity/packageuwp/ConsoleUWP\.appxSources$
^src/tools/lnkd/lnkd\.bat$
^src/tools/pixels/pixels\.bat$
^src/tools/texttests/fira\.txt$

You should consider excluding directory paths (e.g. (?:^|/)vendor/), filenames (e.g. (?:^|/)yarn\.lock$), or file extensions (e.g. \.gz$)

You should consider adding them to:

.github/actions/spelling/excludes.txt

File matching is via Perl regular expressions.

To check these files, more of their words need to be in the dictionary than not. You can use patterns.txt to exclude portions, add items to the dictionary (e.g. by adding them to allow.txt), or fix typos.

To accept these unrecognized words as correct (and remove the previously acknowledged and now absent words), run the following commands

... in a clone of the git@github.com:microsoft/terminal.git repository
on the dev/pabhoj/regex_search branch:

update_files() {
perl -e '
my @expect_files=qw('".github/actions/spelling/expect/alphabet.txt
.github/actions/spelling/expect/expect.txt
.github/actions/spelling/expect/web.txt"');
@ARGV=@expect_files;
my @stale=qw('"$patch_remove"');
my $re=join "|", @stale;
my $suffix=".".time();
my $previous="";
sub maybe_unlink { unlink($_[0]) if $_[0]; }
while (<>) {
if ($ARGV ne $old_argv) { maybe_unlink($previous); $previous="$ARGV$suffix"; rename($ARGV, $previous); open(ARGV_OUT, ">$ARGV"); select(ARGV_OUT); $old_argv = $ARGV; }
next if /^(?:$re)(?:(?:\r|\n)*$| .*)/; print;
}; maybe_unlink($previous);'
perl -e '
my $new_expect_file=".github/actions/spelling/expect/9662bc69102e2e152a062a9060cb63d2742582d3.txt";
use File::Path qw(make_path);
use File::Basename qw(dirname);
make_path (dirname($new_expect_file));
open FILE, q{<}, $new_expect_file; chomp(my @words = <FILE>); close FILE;
my @add=qw('"$patch_add"');
my %items; @items{@words} = @words x (1); @items{@add} = @add x (1);
@words = sort {lc($a)."-".$a cmp lc($b)."-".$b} keys %items;
open FILE, q{>}, $new_expect_file; for my $word (@words) { print FILE "$word\n" if $word =~ /\w/; };
close FILE;
system("git", "add", $new_expect_file);
'
(cat '.github/actions/spelling/excludes.txt' - <<EOF
$should_exclude_patterns
EOF
) |grep .|
sort -f |
uniq > '.github/actions/spelling/excludes.txt.temp' &&
mv '.github/actions/spelling/excludes.txt.temp' '.github/actions/spelling/excludes.txt'
}

comment_json=$(mktemp)
curl -L -s -S \
  --header "Content-Type: application/json" \
  "https://api.github.com/repos/microsoft/terminal/issues/comments/952184317" > "$comment_json"
comment_body=$(mktemp)
jq -r .body < "$comment_json" > $comment_body
rm $comment_json

patch_remove=$(perl -ne 'next unless s{^</summary>(.*)</details>$}{$1}; print' < "$comment_body")
  

patch_add=$(perl -e '$/=undef;
$_=<>;
s{<details>.*}{}s;
s{^#.*}{};
s{\n##.*}{};
s{(?:^|\n)\s*\*}{}g;
s{\s+}{ }g;
print' < "$comment_body")
  

should_exclude_patterns=$(perl -e '$/=undef;
$_=<>;
exit unless s{(?:You should consider excluding directory paths|You should consider adding them to).*}{}s;
s{.*These sample patterns would exclude them:}{}s;
s{.*\`\`\`([^`]*)\`\`\`.*}{$1}m;
print' < "$comment_body" | grep . || true)

update_files
rm $comment_body
git add -u

zadjii-msft · 2023-08-21T15:04:28Z

@PankajBhojwani I'm tempted to close this and resurrect it after #15858 merges. Thoughts?

lhecker · 2023-08-21T15:09:29Z

Oh neat, this PR already got a SVG icon for SearchBox_Regex. We can use that in a future PR!
After #15858 all we need to do to support regex is shove a bool all the way through to TextBuffer and that's it. It already uses a regex searcher (ICU uregex) even for non-regex searches (by setting UREGEX_LITERAL).

PankajBhojwani · 2023-08-21T18:05:17Z

@PankajBhojwani I'm tempted to close this and resurrect it after #15858 merges. Thoughts?

Agreed!

zadjii-msft · 2023-08-21T18:08:31Z

Gotcha. We'll resurrect... shortly 😄

DHowett · 2023-08-21T18:10:45Z

Delete branch?

DHowett · 2023-08-22T21:02:05Z

Ah, the asset was a copy of the Case Search asset

I was keeping the branch around because of the asset, but... yea.

@lhecker @KalleOlaviNiemitalo asked about pathological regeces and cancellation, which we may want to address or at least consider in the followup PR

zadjii-msft · 2023-08-22T21:14:07Z

ACK

Yea we'll need a new svg for that 🤦

PankajBhojwani added 3 commits February 19, 2021 16:19

initial

dfa8164

wrap around

65fba1d

find previous

f7c47a0

PankajBhojwani commented Feb 24, 2021

View reviewed changes

comment

71b83da

KalleOlaviNiemitalo reviewed Feb 24, 2021

View reviewed changes

PankajBhojwani added 4 commits February 24, 2021 09:52

update comment

08f5843

take into account glyph width

09fa066

const

12b5f65

move increment to inside the loop

839af9f

PankajBhojwani and others added 3 commits March 1, 2021 14:21

fix test hopefully?

324a6a3

Migrate spelling-0.0.19 changes from main

0d6631e

Migrate spelling-0.0.21 changes from main

4a45e05

zadjii-msft mentioned this pull request Sep 20, 2022

Epic: Search v2 #3920

Open

20 tasks

zadjii-msft closed this Aug 21, 2023

DHowett deleted the dev/pabhoj/regex_search branch August 22, 2023 21:02

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Allow regex search #9228

Allow regex search #9228

PankajBhojwani commented Feb 20, 2021 •

edited

PankajBhojwani Feb 24, 2021

KalleOlaviNiemitalo Feb 24, 2021

KalleOlaviNiemitalo Feb 26, 2021

PankajBhojwani commented Feb 26, 2021

github-actions bot commented Oct 26, 2021 •

edited

zadjii-msft commented Aug 21, 2023

lhecker commented Aug 21, 2023

PankajBhojwani commented Aug 21, 2023

zadjii-msft commented Aug 21, 2023

DHowett commented Aug 21, 2023

DHowett commented Aug 22, 2023

zadjii-msft commented Aug 22, 2023

Allow regex search #9228

Allow regex search #9228

Conversation

PankajBhojwani commented Feb 20, 2021 • edited

Summary of the Pull Request

References

PR Checklist

Validation Steps Performed

PankajBhojwani Feb 24, 2021

Choose a reason for hiding this comment

KalleOlaviNiemitalo Feb 24, 2021

Choose a reason for hiding this comment

KalleOlaviNiemitalo Feb 26, 2021

Choose a reason for hiding this comment

PankajBhojwani commented Feb 26, 2021

github-actions bot commented Oct 26, 2021 • edited

@check-spelling-bot Report

Unrecognized words, please review:

zadjii-msft commented Aug 21, 2023

lhecker commented Aug 21, 2023

PankajBhojwani commented Aug 21, 2023

zadjii-msft commented Aug 21, 2023

DHowett commented Aug 21, 2023

DHowett commented Aug 22, 2023

zadjii-msft commented Aug 22, 2023

PankajBhojwani commented Feb 20, 2021 •

edited

github-actions bot commented Oct 26, 2021 •

edited