Skip to content

Data flow: Various performance tweaks#2480

Merged
jbj merged 3 commits intogithub:masterfrom
hvitved:dataflow/performance-tweaks
Dec 3, 2019
Merged

Data flow: Various performance tweaks#2480
jbj merged 3 commits intogithub:masterfrom
hvitved:dataflow/performance-tweaks

Conversation

@hvitved
Copy link
Contributor

@hvitved hvitved commented Dec 2, 2019

Pipelines like

[2019-12-02 11:10:19] (378s) Starting to evaluate predicate DataFlowImpl::simpleArgumentFlowsThrough0#fffff#cur_delta/5[2]@1bf604 (iteration 2)
[2019-12-02 11:10:39] (397s) Tuple counts for DataFlowImpl::simpleArgumentFlowsThrough0#fffff#cur_delta:
                      177085  ~0%       {4} r1 = SCAN DataFlowImpl::simpleParameterFlow#ffff#reorder_1_3_0_2#prev_delta AS I OUTPUT I.<2>, I.<0>, I.<1>, I.<3>
                      7851258 ~2%       {7} r2 = JOIN r1 WITH DataFlowImpl::simpleArgumentFlowsThrough0#fffff#join_rhs AS R ON FIRST 1 OUTPUT R.<1>, R.<2>, R.<3>, R.<0>, r1.<1>, r1.<3>, r1.<2>
                      6006273 ~1%       {7} r3 = SELECT r2 ON r2.<1> >= r2.<6>
                      3965088 ~4%       {7} r4 = SELECT r3 ON r3.<1> <= r3.<6>
                      3965088 ~0%       {5} r5 = SCAN r4 OUTPUT r4.<4>, r4.<0>, r4.<2>, r4.<5>, r4.<6>
                      6210    ~161%     {5} r6 = JOIN r5 WITH DataFlowPrivate::ReturnNode::getKind_dispred#ff AS R ON FIRST 1 OUTPUT r5.<1>, r5.<2>, r5.<3>, r5.<4>, R.<1>
                      6210    ~161%     {5} r7 = r6 AND NOT DataFlowImpl::simpleArgumentFlowsThrough0#fffff#prev AS R(r6.<1>, r6.<0>, r6.<4>, r6.<2>, r6.<3>)
                      6210    ~176%     {5} r8 = SCAN r7 OUTPUT r7.<1>, r7.<0>, r7.<4>, r7.<2>, r7.<3>
                                        return r8

are changed into

[2019-11-29 13:00:47] (4649s) Starting to evaluate predicate DataFlowImpl::simpleArgumentFlowsThrough0#fffff#cur_delta/5[2]@7fc688 (iteration 2)
[2019-11-29 13:00:47] (4649s) Tuple counts for DataFlowImpl::simpleArgumentFlowsThrough0#fffff#cur_delta:
                      177085 ~0%     {4} r1 = SCAN DataFlowImpl::simpleParameterFlow#ffff#reorder_1_3_0_2#prev_delta AS I OUTPUT I.<0>, I.<2>, I.<3>, I.<1>
                      776    ~3%     {5} r2 = JOIN r1 WITH DataFlowPrivate::ReturnNode::getKind_dispred#ff AS R ON FIRST 1 OUTPUT r1.<1>, r1.<0>, R.<1>, r1.<2>, r1.<3>
                                     return r2

[2019-11-29 13:00:52] (4654s) Starting to evaluate predicate DataFlowImpl::simpleArgumentFlowsThrough1#fffff#cur_delta/5[3]@e453f8 (iteration 3)
[2019-11-29 13:00:52] (4654s) Tuple counts for DataFlowImpl::simpleArgumentFlowsThrough1#fffff#cur_delta:
                      776     ~0%       {4} r1 = SCAN DataFlowImpl::simpleArgumentFlowsThrough0#fffff#prev_delta AS I OUTPUT I.<0>, I.<2>, I.<3>, I.<4>
                      8667    ~171%     {5} r2 = JOIN r1 WITH DataFlowImplCommon::ImplCommon::Cached::viableParamArg#fff_102#join_rhs AS R ON FIRST 1 OUTPUT r1.<1>, r1.<2>, r1.<3>, R.<1>, R.<2>
                      8667    ~171%     {5} r3 = r2 AND NOT DataFlowImpl::simpleArgumentFlowsThrough1#fffff#prev AS R(r2.<3>, r2.<4>, r2.<0>, r2.<1>, r2.<2>)
                      8667    ~160%     {5} r4 = SCAN r3 OUTPUT r3.<4>, r3.<0>, r3.<1>, r3.<2>, r3.<3>
                      12030   ~162%     {6} r5 = JOIN r4 WITH DataFlowImpl::nodeCand1#ff AS R ON FIRST 1 OUTPUT r4.<1>, r4.<2>, r4.<3>, r4.<4>, r4.<0>, R.<1>
                      9071    ~150%     {6} r6 = SELECT r5 ON r5.<5> >= r5.<2>
                      6210    ~148%     {6} r7 = SELECT r6 ON r6.<5> <= r6.<2>
                      6210    ~176%     {5} r8 = SCAN r7 OUTPUT r7.<3>, r7.<4>, r7.<0>, r7.<1>, r7.<2>
                                        return r8

The dist-compare report for C# shows a notable difference on mono. (Note that the run includes the changes from #2308, as mono is not analyzable prior to that PR.)

@hvitved hvitved requested a review from aschackmull December 2, 2019 12:49
@hvitved hvitved requested review from a team as code owners December 2, 2019 12:49
@aschackmull
Copy link
Contributor

I'm working on a large change to the dataflow libs, which among other things will delete simpleParameterFlow and simpleArgumentFlowsThrough. Could we hold off on this before that's merged?

jbj
jbj previously approved these changes Dec 2, 2019
@hvitved hvitved requested review from jbj and removed request for a team December 2, 2019 19:42
@jbj jbj merged commit 57917be into github:master Dec 3, 2019
@hvitved hvitved deleted the dataflow/performance-tweaks branch December 3, 2019 17:44
max-schaefer pushed a commit to github/codeql-go that referenced this pull request Dec 6, 2019
This brings `DataFlowImpl.qll` and `DataFlowImplCommon.qll` up-to-date with the other languages as of github/codeql#2480.
max-schaefer pushed a commit to github/codeql-go that referenced this pull request Dec 6, 2019
This brings `TaintTrackingImpl.qll` up-to-date with the other languages as of github/codeql#2480.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants