File search can exceed max string length

[baoziv587]

- VSCode Version: 1.5.1

- OS Version:mac OS X 10.11.6

Steps to Reproduce:

1. open file search window (command+p)
2. input something

I can not get any file search result in my project after updating The latest version of vscode(1.5.1).

And i noticed that when i am typing some keyword of file name ,CODE will generate some subprocess do these search job with 80 percent of the CPU cost

Function decodeData(invoked by Class FileWalker) always got same error('vs/workbench/services/search/node/fileSearch.ts').

I did not find out the reason , hope these info is helpful.

[bpasero]

@chrmarti fyi

[shaoshuai0102]

Same problem here. Find processes take nearly 100 percent of my CPU.

[chrmarti]

The error appears to indicate the process ran out of memory. It looks like the find command doesn't always terminate or is spawned multiple times. Not sure if decoding might play a role here.

@baoziv587 could you run time (find -L . -type f | wc -c) in your project folder and paste the output here? That might allow us to assess if it is something with the find command itself.

@bpasero are there known issues with node's built-in string_decoder module, should I be using iconv-lite instead?

[baoziv587]

@chrmarti

the output

672575727
( find -L . -type f | wc -c; ) 3.90s user 21.72s system 108% cpu 23.625 total

[bpasero]

@chrmarti iconv is only used to decode non-utf8 to utf8 and when utf8 is set it falls back to use the node.js decoder. You think this is an encoding issue?

[chrmarti]

Not encoding related then. That string is simply too long.

@baoziv587 Could you also run time (find -L . -type f | wc -l); echo $?? That will tell us how many files your project folder has and what exit code the find command has. This is to better understand why the output of find is so large.

[baoziv587]

@chrmarti

the output

---

3285397
( find -L . -type f | wc -l; ) 2.43s user 19.93s system 103% cpu 21.554 total
0

[baoziv587]

@chrmarti Perhaps because of these floders. If skip these folders ,there are alright

@shaoshuai0102 you also install node_modules with cnpm?

[chrmarti]

Thanks @baoziv587. I will use this report to track the string size problem and #11874 the underlying problem that we retrieve the entire file list before applying the exclusion filters.

[chrmarti]

The underlying problem is fixed in the latest insiders build. Will still address this one.

[baoziv587]

ok,thanks @chrmarti

[roblourens]

Verified by code review