TextContainer GetLocation Throws Exception Occasionally

[guyacosta]

Describe the bug
On occasion there the RulesEngine throws an exception for certain files when trying to determine match locations from the TextContainer.cs file indicated. The issue appears to be a conditional that allows there to be an uneven number of items between the LineEnds and LineStarts lists culminating in a failure in the GetLocation method on line 218. The solution appears to be correcting the boundary limit in the constructor to include '<=' on line 35.

To Reproduce
Source code to reproduce will vary...was primarily python if that helps.
The behavior interestingly was seen while using the NuGet model but not the commandLine one -not sure why but time is limited to investigate but it is also not consistent for reasons unknown i.e. sometimes it doesn't throw an exception.

Expected behavior
Complete analysis for source without throwing an exception

Code Section
File: ..\ApplicationInspector-main\RulesEngine\TextContainer.cs
Class & Method: TextContainer : TextContainer
Line: 35

while (pos > -1)
{
           **if (pos > 0 && pos + 1 < FullContent.Length)**
           {
                    LineStarts.Add(pos + 1);
           }
           pos = FullContent.IndexOf('\n', pos + 1);
     
           if (LineEnds.Count < LineStarts.Count)
           {
                LineEnds.Add(FullContent.Length - 1);
           }
}

and in method GetLocation line 218 where the indexing exception gets thrown

result.Column = index - LineStarts[i];

Screenshots

Additional context
Add any other context about the problem here.

[gfs]

The solution appears to be correcting the boundary limit in the constructor to include '<=' on line 35.

ApplicationInspector/RulesEngine/TextContainer.cs

Lines 26 to 45 in 0e2ea0c

	LineEnds = new List<int>() { 0 };
	LineStarts = new List<int>() { 0, 0 };
	// Find line end in the text
	int pos = FullContent.IndexOf('\n');
	while (pos > -1)
	{
	LineEnds.Add(pos);
	if (pos > 0 && pos + 1 < FullContent.Length)
	{
	LineStarts.Add(pos + 1);
	}
	pos = FullContent.IndexOf('\n', pos + 1);
	}
	if (LineEnds.Count < LineStarts.Count)
	{
	LineEnds.Add(FullContent.Length - 1);
	}

Have you tested that change to confirm it fixes this issue? The 'pos' variable is 0 indexed but it is being compared to a Length which is essentially 1 indexed. The check for pos + 1 < Length ensures that we don't add a Line Start after the file has ended, a position equal to length is beyond the length of the 0 indexed string.

I do think the issue is on this line though. I have reproduced that if the first character of your file is \n then it will skip that LineStart. I haven't gotten it to crash from this, but it does desync the line starts and ends.

[guyacosta]

Yes. It works -the reason is that the new line should be added to balance the ends and isn't. I wanted to submit the fix but having some cert issue doing so. I think I can use another system actually and will try that.

[guyacosta]

Reverified locally. Running unit tests now and will submit. I also added a try/catch around the access lines in the GetLocation method because worst case is an empty Location. Note the logger is not passed this far down into the code.

[gfs]

I pushed the change to fix the starting with \n bug. I'll leave this open until I check your PR as well. Please make sure to rebase onto main to get the one line change I just merged.

[guyacosta]

Getting a 403 error on trying to push a branch using this account. I'll rebase first (tried to push in-between) but any suggestions welcome. Probably been too long and I'm missing some simple thing lol.

[gfs]

You won't have access to the Microsoft repo so you need to make a fork and then make a branch on that fork and then open a PR from that branch.

[gfs]

Can you still reproduce this after I the change I pushed?

[guyacosta]

Checking now...

[guyacosta]

Looks fixed!