Bad performance of textTagEndBlock4 in SwiftParser

[ynleborg]

Since version SRU2016-7.8.5 there is a performance problem in the implementation of textTagEndBlock4 in SwiftParser. Processing increased from 3 seconds to 3 minutes for 14MB file. Profiling tools show, that most time is spent in substring method. We internally reverted implementation to SRU2015-7.7 and it fixed issue.

[zubri]

Hi,

Thanks for the feedback.
We see no significant changes in the usage of substring calls in that method, neither in the whole parser.

Can you provide more details on how you are reading and parsing the 14MB file? Are you using the RJEReader?

[ynleborg]

I have prepared test that reproduces the bug: #6

On my computer shouldParseLargeFile runs 12 seconds on SRU2016-7.8.5 and 0,187s on SRU2015-7.7.
The number of processed records is the some in both cases: 30971.

[zubri]

This has been fixed to achieve previous performance. I've removed the substring call and refactored the tag start detection. It will be included in the upcoming SRU2017 release in October.

However, notice the large statement that you are using to produce the issue is not valid. And input MT940 can have at most 2K characters.

Also, the parser from 2015 7.7 you are using has a bug. It does not accept ':' as starting content in the second line of a field.