[RFC 0004] Replace Unicode Quotes

[layus]

Nix uses unicode glyphs to quote strings and paths in its output. This RFC
proposes to use only ASCII " and ' for quoting purposes in strings printed
during evaluation.

Rendered @ https://github.com/layus/rfcs/blob/master/rfcs/0004-replace-unicode-quotes.md

[zimbatm]

👍 overall. This RFC could be ready for general feedback quite quickly.

[zimbatm]

You can add me as a co-author if you want

[zimbatm]

This point could be more compelling with one or two examples

[zimbatm]

Are there specific issues to point to? Are there specific fonts that's don't have the character defined?

[zimbatm]

It should also not modify derivation results

[zimbatm]

Is nixpkgs part of the scope of this RFC or just nix?

[zimbatm]

typo: optiosn -> options

[layus]

@vcunat Could you help here by providing more references to your comment NixOS/nix#947 (comment) ? Is there some ML thread, IRC discussion or github comment I could refer to to make this point ?

[layus]

@zimbatm update to fix/improve on all you comments.

[zimbatm]

@layus thanks, do you mind to push further commits without rebasing? that helps keep me track of the changes over time.

[zimbatm]

I think it's ready for wider review. What do you think @layus ?

[layus]

@zimbatm You are the first guy I ever met to be against PR squashing (except me). Everything returned to normal, I restored the old commit and the update.

And yes, I think it is ready for wider approval :-)

[zimbatm]

Yeah, in this context I think it makes sense because there is only one logical commit produced at the end, which can be created with the "Squash and merge" button. A lot of the rebasing culture came before that button appeared.

[vcunat]

@layus: I can't see any comment of mine on that MR. What point do you want to support?

[vcunat]

AFAIK the main usage of non-ASCII (quotes) is currently in nix repo, probably mostly coming from /cc @edolstra. @domenkozar seemed to be confident that we should get rid of them. I've got personally no hard feelings about this. (These are really simple unicodes.) EDIT: but the reasoning seems well-founded, so I'm 👍 generally in the end.

[layus]

Yes, sorry, I meant @domenkozar. Got my brains messed up apparently :). Why did he(you) feel confident that we should get rid of unicode quotes ?

[domenkozar]

Because our package manager is meant to work in many different environments and it's not worth the trouble getting terminal encoding working over some pragmatism and just using ASCII.

[spinus]

@domenkozar on a practical side, I think most distros which nix can run on, already use UTF-8. Even if not, I think (not sure 100%) despite the encoding you can have non-ascii, non-utf-8 characters in path anyway so this situation needs to be supported anyway, is that right?

[avnik]

Upvote! basic package management, pkgs querying and early boot stages should be 7bit clean

[domenkozar]

@spinus you could have other encoding than utf-8 for filesystem and terminal, then our utf-8 encoded prints would fail. We could detect what terminal encoding is used ($LANG) and use that for printing, hoping it would fix macOS. But then, why all the fuzz?

[spinus]

I would vote exactly for what you proposed with detecting $LANG.

But then, why all the fuzz?

I think that's good question and I assume that's why people started RFC. But as far as we have that we can document (in a central place) why certain decision was made (to not discuss it too often :-) )

[zimbatm]

Is it worth the added complexity though? Regardless of the $LANG discussion, it would still cause friction when copy-and-pasting on debug.

[garbas]

👍 from my side, but I would like to know why @edolstra chose those characters in the first place. Maybe there is a reason that we are not aware of.

Another thing: Would this maybe break Hydra? The view where logs are collapsable, maybe somewhere else...

[benley]

The collapsible part of hydra's log viewer is already broken for unrelated reasons; see NixOS/hydra#240. This would not make it worse, as the feature does not rely on unicode quoting.

[vcunat]

Replacing them is relatively easy, but the IMO the largest blocker now is missing reaction from @edolstra.

[edolstra]

The main motivation - that some terminal emulators handle UTF-8 badly - doesn't seem very convincing to me. I mean, come on, it's 2017. If a terminal emulator can't handle UTF-8 today, it's time to fix or replace it. Nix is hardly the only program that does Unicode. FWIW, I've never had problems with this in Konsole.

However I don't feel super strongly about this.

[domenkozar]

@edolstra it does bring a good point, why doesn't it work on macOS?

[esoeylemez]

Many programs in order to handle Unicode properly need to have at least LC_CTYPE set, which requires glibcLocales, which in turn requires 109 MiB of extra closure. While I do agree that everything should just be Unicode-ready these days, that's not the way it is. I'm in favour of switching to ASCII for now. Yes, "it's 2017", and we still suck at encodings.

That said, I don't run into any problems with Unicode quotes in practice. We're building our containers using Nix, but we're not actually adding Nix itself to them. And if we were, my first impulse would be to write a function that generates a narrower variant of glibcLocales, because most of the time only one or two locales are needed.

[vcunat]

@esoeylemez: the package has a parameter that you can override to the desired list of locales, and there's even an explicit option in nixos: http://nixos.org/nixos/manual/options.html#opt-i18n.supportedLocales

[zimbatm]

I think the main reason should be changed to being a (small) usability issue.

1. When nix outputs a path, usually I copy, type vim and paste it. I have to make sure to not select the outer quotes because then it's not a valid path anymore. It means I have to be more precise in my selection.

2. In some shells (like Hyper) the unicode quote is not a word delimiter. It means that in the following example:

cat <<FOO
a.   ‘0ff098i8dlr6frks67ik0kbc281c6j8lkb6v0y33iwqv45n233q3’
b.   '0ff098i8dlr6frks67ik0kbc281c6j8lkb6v0y33iwqv45n233q3'
FOO

(a) selects the outer quotes, (b) selects only the hash.

Arguably I could also use a better terminal :)

[esoeylemez]

@vcunat: Thanks, I didn't know that.

All in all I'm still in favour of replacing them though. The UX issue @zimbatm mentioned has been irritating enough that I had actually made a pull request to replace the quotes in one specific location (when hash matching fails), because copying a hash is very awkward right now. It got rejected, because (understandably) it would make the quotes inconsistent.

My view is this: there doesn't seem to be any advantage of Unicode quotes other than aesthetics. But they do come at a practical expense. If the maintainers agree, I'm willing to do the actual work of replacing them everywhere, except perhaps in docs, where I believe the issues don't apply.

[danbst]

@edolstra
Attaching a GIF on how double-click selecting works in urxvt with 1) unary quotes, 2) Nix quotes:

[layus]

@zimbatm I have the feeling that the discussion covered the important aspects, and that we reached a good understanding or the issue.

To summarize, this RFC proposes an impacting change to the codebase to get rid of annoyances and occasional bugs. Removing unicode quotes is the simplest way to fix these annoyances and potential new issues.

Not implementing this RFC means that we can keep these nice quotes. It would state our will to support unicode correctly in the nix ecosystem, by fixing issues, providing patches and writing manuals on how to type/enter these characters.

We all need to pick our fights, and I am ready to drop aesthetics and unicode integration to spare time for other tasks. This process itself is taking some energy :-).

Lookink at the discussion, people are mostly in favor of the RFC and no strong objections were raised. Some even proposed to implement it. Let's take a final decision, and start with another RFC, I see that there are more pending :-).

[shlevy]

Not sure what the final decision process is, but if it's a vote: thumbs up or thumbs down on this comment.

[benley]

One last thing I'll add: This proposal does not need to be a referendum on Unicode support in the Nix ecosystem in general. It's just about the console output of the commandline tools, right?

[zimbatm]

Yup let's wrap this up. The main motivation is usability of quoting so it doesn't preclude from adding emojis to the output later on if we want ;)

…

On Wed, 12 Apr 2017, 17:58 Benjamin Staffin, ***@***.***> wrote: One last thing I'll add: This proposal does not need to be a referendum on Unicode support in the Nix ecosystem in general. It's just about the console output of the commandline tools, right? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#4 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAAMsAiorZFqo0KLoz77p4EC5mTdo4d8ks5rvQKbgaJpZM4MhuLj> .

[esoeylemez]

Great. I'd like to wait until Saturday, and if the number of upvotes on shlevy's comment is higher than the number of downvotes, I'll implement the change and send a pull request.

[vcunat]

I think there's a good consensus already, and the RFC has been open for almost a month, giving enough time to notice it, but we can wait a couple more days. EDIT: I don't connect this at all with reducing unicode support in nix(pkgs).

[aneeshusa]

There looks to be a lot of support for this and it has been almost another month - @esoeylemez any progress on a PR to implement this?

[esoeylemez]

Sorry for the delay. Working on it right now.

[grahamc]

As of now, the vote is 13 thumbs-ups and 0 thumbs-downs.

[grahamc]

^ I added this comment since votes can be changed later :)

[Mic92]

This have been now implemented in NixOS/nix#1495