Add feature to hide all non-printable characters

[Gitoffthelawn]

From what I can tell, there is no way to hide all non-printable characters.

I've searched the web for ways to accomplish this in Notepad++, and it seems to be commonly desired functionality.

Can it be implemented?

[sasumner]

What's a "non-printable" character?

seems to be commonly desired functionality

Can you cite some references for that?

[Gitoffthelawn]

What's a "non-printable" character?

All ASCII characters below hexadecimal 20 (decimal 32), with the exception of hex 09 (dec 9), 0A (dec 10), and hex 0D (dec 13). Also hex 81 (dec 129), hex 8D (dec 141), hex 8F (dec 143), hex 90 (dec 144), and hex 9D (dec 157).

Notepad++ represents them all as 2-3 character inverted text control strings (many of which are references to ancient serial communication protocols from 40+ years ago).

The best way to hide them is usually to hide all hex 00 (nulls), and show all sequences of 1 or more non-printable characters with a single space (hex 20).

Can you cite some references for that?

Yes, if that will result in getting it implemented. :)

I performed a bunch of web searches looking for this Notepad++ functionality, and I'll I found was other people disappointed in the lack of this feature. IIRC, I also found an old Notepad++ issue (on Sourceforge) from about a decade ago that said this would be resolved soon as the project was no longer going to use Scintilla! ;)

[sasumner]

What's a "non-printable" character?

All ASCII characters below hexadecimal 20 (decimal 32), with the exception of hex 09 (dec 9), 0A (dec 10), and hex 0D (dec 13). Also hex 81 (dec 129), hex 8D (dec 141), hex 8F (dec 143), hex 90 (dec 144), and hex 9D (dec 157).

By that definition, why aren't the following also in the list, (decimal) 127, 128, 130-140, 142, 145-156, 158-255 ?

The best way to hide them is usually to hide all hex 00 (nulls), and show all sequences of 1 or more non-printable characters with a single space (hex 20).

As one person's data point, I'm not convinced that hiding data from the user is the best thing to do.

I performed a bunch of web searches

That really isn't citing references unless you share the results as explicit web links. :-)

[Gitoffthelawn]

By that definition, why aren't the following also in the list, (decimal) 127, 128, 130-140, 142, 145-156, 158-255 ?

Because all of those characters are easy to visually represent and are printable. See http://www.asciitable.com/

As one person's data point, I'm not convinced that hiding data from the user is the best thing to do.

No offense, but why do you believe I need to convince you? I'm not asking for it to be a default, just an option. Similar to all the other options already implemented under the View->Show Symbol menu. Note that, by default, Notepad++ does currently hide some data. You have to explicitly enable an option to tell it not to hide some characters.

That really isn't citing references unless you share the results as explicit web links. :-)

Yes, of course. If I'm going to spend 20 minutes of my time redoing all those searches, I would like some assurances that it will result in this getting implemented. Otherwise, there is no reason for me to spend that time.

[sasumner]

Not sure ASCII table data is any more relevant in a modern setting than "ancient serial communication protocols from 40+ years ago". First half of ASCII table is equivalent to UTF-8, second/upper half of the the table, well, might be best forgotten. :-)

No offense, but why do you believe I need to convince you?

None taken; I'm just a simple member of the Notepad++ development team.

Notepad++ does currently hide some data

Yes, but only characters that are considered "meta" to a text editor.

If I'm going to spend 20 minutes of my time

This is an incredibly incendiary statement, as anyone implementing a change would burn that same amount of time at a minimum just to administrate it.

I was particularly interested in the ref. regarding "old Notepad++ issue (on Sourceforge) from about a decade ago that said this would be resolved soon as (sic) the project was no longer going to use Scintilla!"

The best course of action might be to write some sort of script that can filter out unwanted characters entirely from data, that way no one could possibly be misled by just not seeing it. This could be done easily with a scripting plugin or an external filter. It presumes you "own" your data, however; it may not be practical if you work with data from sources you don't control.

I'm going to close this for now, but I will mark it as depending upon Scintilla. What that means is that if Scintilla can do it, Notepad++ can do it also, but if Scintilla can't do it, neither can Notepad++. At present I'm not going to spend any time researching whether or not Scintilla can currently do it; if the original poster or someone else wants to do that research and report back here , then that's fine.

[celerswift]

This is the second time i've seen this project be rude to people asking for help about something they need. I would suggest a course in being polite and trying to understand what your users are asking for instead of telling them what's best for them and being rude.

I am uninstalled Notepad++ for the last time and I will not be recommending it to anyone who asks.

[alankilborn]

This is the second time i've seen this project be rude to people asking for help about something they need. I would suggest a course in being polite and trying to understand what your users are asking for instead of telling them what's best for them and being rude.

I've reread the above conversations and didn't see anything that I'd consider impolite.
Can you be more specific?

[celerswift]

This is the second time i've seen this project be rude to people asking for help about something they need. I would suggest a course in being polite and trying to understand what your users are asking for instead of telling them what's best for them and being rude.

I've reread the above conversations and didn't see anything that I'd consider impolite. Can you be more specific?

You are confirming the complaint by circling the wagons.

[alankilborn]

@celerswift

You are confirming the complaint by circling the wagons.

And you are speaking in riddles.
(Is that sort of reply rude? I surely hope not)

I suppose, without further information, there's nothing left to be said.
Good luck with your next choice of text editor.

[alankilborn]

Now that I have a bit more time, I'm reading all of the above exchanges with a more critical eye.
While I'm still not getting a rudeness vibe, I think there is probably something that can be done here.

I'm thinking that perhaps the feature request is a variant on a "Show all characters" option.
Perhaps what is needed is "categories to show" and by inference, if you aren't showing them, you're hiding them.

So @Gitoffthelawn 's request in this issue is merely one of the "categories".
The categories could be shown/hidden by a list of checkboxed categories -- if you want the category shown, tick the checkbox; if not untick it.

Some categories could be:

• simple space, tab (existing N++ "whitespace" show/hide)
• line-ending characters (also currently an existing "category" in N++)
• @Gitoffthelawn 's list of "ancient ASCII characters"
• invisible UTF+8 characters (NBSP, etc)

The list could go on and on. Maybe there's even one or more user-definable custom categories.

To me this feels like it could all be incorporated into a solution for #8284

[alankilborn]

@Gitoffthelawn

Notepad++ 8.5 will feature this new command:

Is it your feeling, in the context of this current issue, that when that menu item is un-checkmarked, that these characters should not be shown?:

[alankilborn]

I did some looking into this issue, and have concluded that it isn't really practical to implement, given the way in which Scintilla seems to always insist on displaying the control characters in question. Some supplementary information is in this Community thread, starting with the posting at the indicated time: https://community.notepad-plus-plus.org/topic/14045 Feb 13, 2023, 7:51 AM

[ozone10]

@alankilborn
set representation to space or other whitespace and use SCI_SETREPRESENTATIONAPPEARANCE with SC_REPRESENTATION_PLAIN

[alankilborn]

set representation to space or other whitespace and use SCI_SETREPRESENTATIONAPPEARANCE with SC_REPRESENTATION_PLAIN

Thanks @ozone10 for the tip.

I tried it and it works except of course it leaves the width of the space visible at the location of the control character. I suppose this is "better than nothing", but it doesn't encourage me enough to continue looking at this problem, as I'd think users would not like this extra space?

[ozone10]

@alankilborn
try to set representation to zero-width space with non-printing characters hidden

{"\xE2\x80\x8B", "ZWSP", "U+200B"}, // U+200B : zero-width space

[alankilborn]

try to set representation to zero-width space with non-printing characters hidden

Ah, another good call -- that works.

Luckily, these things are both "on the same side" -- meaning, if a change fulfilling this issue is completed, it marries well with the existing Show Non-Printing Characters.

[Gitoffthelawn]

@Gitoffthelawn

Notepad++ 8.5 will feature this new command:

Is it your feeling, in the context of this current issue, that when that menu item is un-checkmarked, that these characters should not be shown?:

Yes, that looks good. Thank you!

The ability to add a keyboard shortcut and a toolbar button to toggle this setting will be most appreciated.

I don't think we've met, and I apologize if you introduced yourself earlier and I missed it. It looks like your GitHub account is relatively new, and was created around the same time that @sasumner suddenly disappeared from GitHub. Are you @sasumner under a new moniker?

[alankilborn]

Yes, that looks good. Thank you!

Don't get too excited; the control characters aren't part of Show Non-Printing Characters currently -- it was just an idea I had, that since they (control chars) are also non-printing, that they really belong as part of the feature enabled by the menu item I highlighted.

sasumner under a new moniker?

You flatter me. Scott was a BIG contributor here; I'm just starting with baby steps as C++ is new to me, to contribute some code to N++ (maybe 3(?) PRs accepted into master in the last few weeks -- yay!).

Maybe this control chars will be another one; have to do more digging into the complexity of such a code change. If it appears to be too large, I will pass on it and maybe someone else will pick it up.

[alankilborn]

I've tried changing some code to support the desire of this issue, but it appears to be too much coding (for me) to get it working fully. I'm definitely sticking to "smaller" changes in what I attempt. Perhaps someone else will come along and take interest in supporting this issue.