Support Javascript! #546
Support Javascript! #546
Conversation
Borrow some of mozsearch's analysis dumping by parsing JS with esprima.
| @@ -12,6 +12,11 @@ apt-get -q update \ | |||
| llvm-3.5 libclang-3.5-dev clang-3.5 \ | |||
| curl | |||
|
|
|||
| # Install newer node. | |||
| apt-get remove -y nodejs | |||
| curl -sL https://deb.nodesource.com/setup_6.x | bash - | |||
There was a problem hiding this comment.
Not a blocker for this PR, but we should see if we can get the various curl-to-root-shell commands out of here. Since we've moved from a proper VM to Docker, our containment is worse.
|
FWIW, using hashes probably won't get us anything and might even get us worse performance by sticking a zillion files in one dir. The reason we use hashes on C++ is that a given file can end up with multiple effective contents due to preprocessor directives, so we throw both the path and the preprocessed contents into the hash. |
|
With js as the only plugin and no build run, mozilla-central completes in 44 minutes. I'll be rolling in another update to this which reflects what it took to get that working... |
| @@ -0,0 +1,81 @@ | |||
| const path = require("path"); | |||
| const fs = require('fs'); | |||
There was a problem hiding this comment.
Eventually we should standardize on a kind of quotes. And use eslint. I have a good default config in Fathom.
|
This looks really good. I'm looking forward to it landing! Give me a poke once you've had a chance to look at my comments. |
Differentiate between global props and qualified props whose first segment could not be found.
|
r+! |
I port over https://github.com/bill-mccloskey/mozsearch/blob/master/js-analyze.js of searchfox.org into DXR.
This plugin splits javascript identifiers into "vars" and "props," where "var" refers to local variables and "prop" refers to either global variables (i.e. properties of the global object) or properties of objects/classes.
At post_build, the plugin calls a node program that walks the source tree, looking for unignored .js files (nb. if we merge #535 then I would not have to reimplement this; the Python could give the node these paths), and dumps JSON lines for each file in a mirrored directory tree in the temp folder.
At index the Python reads these file dumps to drop needles and refs to ES.
I intend to run this plugin on mozilla-central in the near future to see how the output looks and how long it takes (perhaps the organization of temp files could borrow from clang's version and use hashes).