CSV lexer

[zufuliu]

https://www.rfc-editor.org/rfc/rfc4180
https://edoceo.com/dev/csv-file-format

[zufuliu]

A rainbow lexer proposed in #543 will be added, feel free to shared your ideas.

[Matteo-Nigro]

Hi, cross-styling proposal: if automatic delimiter detection (although still the best thing for convenience) were to become complex or impact performance, could one think of creating an inputbox that asks the user when loading a CSV to specify the delimiter? Besides, Excel itself does this.

[zufuliu]

Not sure, if the dialog is added, Scheme -> CSV File also need to be added to open dialog. discussion can moved into #545.

[zufuliu]

Styling code in lexer is finished after commit 25e9425, remaining work is detect delimiter and quotation characters.

[zufuliu]

Not plan to implement delimiter auto detection, instead added Scheme -> CSV File menu.

[Matteo-Nigro]

As I suggested, I think it is the simplest solution. Good work @zufuliu!

[zufuliu]

Another useful option (for tab or space delimited file) is "Treat consecutive delimiters as one.", maybe we can add it too.

[Matteo-Nigro]

Certainly it could be useful in some contexts. 😉

[zufuliu]

Option "Treat consecutive delimiters as one." added by e5d546e.

[zufuliu]

Pipe delimiter and "Escape with backslash" option added by e1fc62c.

[Matteo-Nigro]

Excellent! A perhaps trivial idea is for the field for the custom delimiter to be deactivated and only activated when the Radiobutton "Other" is selected. The interface is very clear but someone could mistakenly write in the field but apply a different delimiter 😂

[zufuliu]

Fixed by 0ebb70a.

[zufuliu]

cd90364 added "None" option for text qualifier, it implies backslash escape (e.g. MySQL CSV).

[meteorquake]

This is very good and useful. When I saw the feature title I just thought it wouldn't be much different from just using VBS or JS mode, but now I see the screenshots with the coloured columns it will be a really great option to have :)
Some extra suggestions -

• Some files of this kind don't have linefeeds for the column repeat but use the column delimeter since it is data for which the application knows how many columns to expect. In this scenario you would have a column repeat number so e.g. if you entered 8 then column 9 would be back to colour 1, as would 17 etc. Obviously you might not want to view huge files of this kind given they are on a single line.
• Often I have JS data laid out as a long array with LFs at regular intervals so a bunch of n elements would represent an item precisely resembling the examples, that this will be useful for, and the delimiter would be a comma. However the JS delimiter methodology is more complex than commas with a simple quote symbol, since in JS you have single and double quotes, and array [] and hash-array {} openers and closers. For example, you would want the following line in a JS data array to be treated as 5 columns for colouring (a b c d e):
  'a', 'b', ['c1','c2'], {d1:d2,d3:d4}, "e1,'e2',e3",
  Perhaps the other languages might benefit too from some intelligent detection of where columns change, such as with an html table, where <td marks a column break and <tr a colour restart, and LFs ignored for colouring, as it would be very useful to be able to view all such things in a similar way to the example screenshot provided.
  David

[zufuliu]

You can use JSON for 'a', 'b', ['c1','c2'], {d1:d2,d3:d4}, "e1,'e2',e3".
Dynamic column count can be implemented by updating following code:

notepad2/scintilla/lexers/LexCSV.cxx

Lines 79 to 82 in 3fb033a

	++initStyle;
	if (initStyle == SCE_CSV_DELIMITER) {
	initStyle = SCE_CSV_COLUMN_0;
	}

[zufuliu]

Close as completed. not interesting on dynamic column count as it's nature to count with 10.