Go: extractor: do not store intermediate values in long string concatenations #15865
Conversation
When we see "a" + "b" + "c" + "d", do not add a
row to the constvalues table for the intermiediate
strings "ab" and "abc". We still have entries for
the string literals ("a", "b", "c", and "d") and
the whole string concatenation ("abcd").
|
I'm thinking about how to test this. Should I add a check that we don't have a string with value "ab" in a db for a go file containing "a" + "b" + "c"? I feel like that isn't quite testing the right thing. Or could we have a pathological example and check the db isn't too big? |
|
The former sounds right to me |
|
DCA shows nothing interesting, so I'll do a QA run to see if that turns up any other repos with this pathologically long string concatenations. |
owen-mc
changed the title
|
QA results looked broadly fine. I saw one case where " Stringpool size measured as 271614442" was replaced by "Stringpool size measured as 11567514" (23 times smaller), which is obviously encouraging. There are three alert changes I should look into - they may just because the base for my comparison wasn't quite right. |
|
I looked into one of the alert changes, which was two extra alerts for I also started looking at the other alert change, which is a lost result for |
|
I investigated the second alert change some more. It is intermittent, though it did happen on QA and the first time I ran locally. The flow doesn't follow a method call because there is no parameter node defined, which is because there is no link between the Function (object/entity) and its declaration. This must come from a problem during extraction. There was a problem extracting the method I don't think this should block this PR, as it seems unrelated to the changes made here. At worst we have perturbed something non-determinimistic which causes this to be a problem more often. |
When we see "a" + "b" + "c" + "d", do not add a row to the constvalues table for the intermediate strings "ab" and "abc". We still have entries for the string literals ("a", "b", "c", and "d") and the whole string concatenation ("abcd").
This was inspired by
golang/go-frontend, which is a somewhat pathological case with a concatenation of 7098 literals of 60 characters each. The db goes down from 2.5GB to 170MB.