Better handling for DOS line endings in ASCII readers

[markcmiller86]

Is your feature request related to a problem?

We have many readers that read ascii files. We typically read them using STL istream objects. However, our parsing logic is not necessarily able to deal with DOS line endings (\r\n) in all cases. We should audit and improve this situation. Instead of using ifstream::getline() directly, we should use a VisIt defined wrapper method for this that detects such cases and removes the offending characters from whatever gets handed back to the plugin parsing logic.

I encountered cases where NASTRAN reader was not behaving as expected with DOS line endings present. It was doing a ton of extra string to number conversion attempts on empty strings because it was failing to see the end of a line.

Is your feature request specific to a data set?

This is specific to ASCII file formats. Any such formats whose files are produced in Windows systems can have DOS line endings. We should be able to support them transparently.

Describe the solution you'd like.

Create a line reading method that ensures all plugin logic only ever sees "normal" line endings in strings read from files. This method can then be used to read files in all ASCII plugins.

Describe alternatives you've considered.

We could instead support a tool that removes DOS line endings from files. But, that seems less useful.

[brugger1]

We should check with @biagas in case the behavior is different on Windows.

[biagas]

I think @markcmiller86 changes to look for '\r' will work if the file was created on Windows and read on linux.
But what about the reverse? Or created on Mac and read elsewhere (and the reverse)? On MacOS are line endings '\n' like on linux?

[markcmiller86]

I think @markcmiller86 changes to look for '\r' will work if the file was created on Windows and read on linux.
But what about the reverse? Or created on Mac and read elsewhere (and the reverse)? On MacOS are line endings '\n' like on linux?

If you are talking about, for example, this code

visit/src/databases/NASTRAN/avtNASTRANFileFormat.C

Lines 1083 to 1100 in 0b087f6

	avtNASTRANFileFormat::ParseLine(vtkIdType *verts, char *line, int start,
	int count)
	{
	char *valstart = line + INDEX_FIELD_WIDTH * start;
	char *valend = valstart + INDEX_FIELD_WIDTH;
	char val_t;
	int i;
	for (i = 0; (i < count) && (*valstart != '\0') && (*valstart != '\r'); i++) {
	if(*valstart == '+' || *valstart == '*')
	{ // At blank continuation element
	valstart += INDEX_FIELD_WIDTH;
	valend += INDEX_FIELD_WIDTH;
	}
	val_t = *valend;
	*valend = '\0';
	verts[i] = Geti(valstart)-1;

That logic will continue marching through the line until it hits either the null char or the \r char. So, I think that means in this paritcular case anyways, it works for files with either kind of line ending on any machine.