Skip to content

Conversation

asgerf
Copy link
Contributor

@asgerf asgerf commented Apr 2, 2020

DataFlow::SourceNode was part of a weird recursive SCC involving AMD, Angular, lazy-cache, HTTP, and LodashUnderscore.

  • The contributions from HTTP and LodashUnderscore were no-ops as the charpred requires that this is a SourceNode. These have been removed.
  • AngularJS contributed to SourceNode in a way that depended on angular() whose return type is SourceNode. Rewritten to use Import instead.
  • AMD and lazy-cache both contributed to Import in a way that depended on SourceNode, and the above changes to Angular means SourceNode depends on Import. Rewritten to rely on AST or the local data flow relation (which does not depend on SourceNode).

Full evaluation on smoke-test and security suite on nightly look good after finally giving up and using a pragma[noopt] in lazy-cache.

@asgerf asgerf added the JS label Apr 2, 2020
@asgerf asgerf requested a review from a team as a code owner April 2, 2020 22:54
@asgerf
Copy link
Contributor Author

asgerf commented Apr 3, 2020

@esbena do you remember why the AngularJS model needs string literals to be SourceNodes? mayHaveStringValue already tracks string constants locally so I was wondering if we could just use that instead, but I couldn't immediately see where it was used.

@esbena
Copy link
Contributor

esbena commented Apr 6, 2020

I think @max-schaefer is the author of this particular feature, perhaps he remembers some subtlety we are missing?

Hisorically, I think the feature is a legacy thing from when we introduced SourceNode, and as a result initially lost the default-flow of string literals.

@asgerf
Copy link
Contributor Author

asgerf commented Apr 8, 2020

Re-run of the slowest slugs without DPMs look good as well.

@asgerf
Copy link
Contributor Author

asgerf commented Apr 8, 2020

@esbena could I ask you for a review?

Copy link
Contributor

@esbena esbena left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. That is some nice detective work.

It seems easy to accidentally re-introduce these costly cycles again, AFAIU you are not breaking any negative cycles in these refactorings - or are you?

I have two proposals:

  • document the places where it is tempting to undo your current improvements (see inline comments)
  • construct a sneaky test that fails if we reintroduce a bad SourceNode/Import dependency - if possible. (The test could either change analysis behaviour or cause a compilation error due to non-monotonic recursion.)

We are still waiting for Max' reply to "perhaps he remembers some subtlety we are missing?" btw.

result instanceof DataFlow::ValueNode
}

private
DataFlow::Node getFactoryNodeInternal() {
result = DataFlow::valueNode(getLastArgument()) or
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we want to add a warning here as well?

@max-schaefer
Copy link
Contributor

perhaps he remembers some subtlety we are missing?

I believe @esbena's answer is correct: this was done to make up for lacking string-literal tracking. Fine to remove it now.

@asgerf
Copy link
Contributor Author

asgerf commented Apr 8, 2020

Excellent idea with the test case. I've added a test that fails if the domain of SourceNode depends on SourceNode.flowsTo which is the root cause of the Import/SourceNode dependency being unwanted.

I'll look into removing the string literal source node from the Angular model, but given the good evaluations so far I think this is safe to land.

esbena
esbena previously approved these changes Apr 8, 2020
@semmle-qlci semmle-qlci merged commit 404f722 into github:master Apr 8, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants