qqww is not documented

[AlexDaniel]

qqw is searchable, but qqww is not.

[dha]

A quick search doesn't turn it up in doc at all, so that might be why it's not searchable...

[AlexDaniel]

That's… a good remark.

It has to be documented here: http://doc.perl6.org/language/quoting

[zoffixznet]

Some pointers for those willing to write the docs: It's :ww in here: https://design.perl6.org/S02.html#Adverbs_on_quotes

And here's an example difference. Note how 'hello world' is one term:

<Zoffix> m: .say for qqw<'hello world' test>
<camelia> rakudo-moar b348f2: OUTPUT«'hello␤world'␤test␤»
<Zoffix> m: .say for qqww<'hello world' test>
<camelia> rakudo-moar b348f2: OUTPUT«hello world␤test␤»

<AlexDaniel> m: .say for <'hello world' test>
<camelia> rakudo-moar b348f2: OUTPUT«'hello␤world'␤test␤»
<AlexDaniel> m: .say for «'hello world' test»
<camelia> rakudo-moar b348f2: OUTPUT«hello world␤test␤»

[jonathanstowe]

Probably it's merge #376 with this one

[AlexDaniel]

@jonathanstowe or maybe not. Let's get qqww documented without trying to think about a huge issue with other quoting adverbs (because «» is very common).

[dha]

So, I'm taking a look at this, and I think there's a thing or two that may need to be resolved before documenting.

Least important: Searching. I think this needs to be documented, even if we aren't clear on how to make it searchable yet. Also, IMO, making <> and <<>> (using Texas here to avoid possible rendering issues) searchable - when we get to that - has slightly more priority than making qqw and qqww searchable. I may, of course, be completely wrong about that, but I think more people would be searching for the operators than the Q language versions.

Mid-important: Should the difference between :w and :ww be documented by itself, or can we just describe the difference under qw and qqw? Or do we need separate sections for every combination of q's and w's?

Most (relatively) important: When we talk about :w being quote-preserving, how do we describe what counts as a quote character? I guess we can leave that question unanswered, but, to be honest, that makes me uncomfortable. I suppose we could do that as a short term solution, but it should be eventually addressed.

Side note: Should we tweak S02 so that it reflects the actual behavior, instead of :w and :ww being switched, as it stands?

FWIW, I took a quick swipe at describing :w and :ww in the qw and qww sections. Only a first pass, so probably not quite up to par, but it's a start: https://gist.github.com/dha/409794ce0679da81507eaf2602d39374

[jonathanstowe]

Right I'll add qqww ( «») in the first instance as it will take a bit longer to test all the speculated adverbs.

[jonathanstowe]

@dha I'm going to add qww and qqww quickly now to get this moving, and move « » from qqw to qqww (so the search is correct,) improvement will probably be required.

[dha]

So, for future reference, apparently, the answer to "what counts as a quote" can be found in Grammar.nqp, line 3518-3527 (i.e. the entries under "proto token quote { <...> }")

Now, all we have to figure out is, when we get around to adding that bit of information to the docs, does it go in under the q[q]w[w] entries, or somewhere else?

psch++

[AlexDaniel]

@dha I don't think that what you've found is relevant in this particular case.

Code:

say „hello world” # works OK, no problem with such quotes

Result:

hello world

But this:
Code:

.say for qqww<foo „hello world” bar> # no quote protection here!

Result:

foo
„hello
world”
bar

In other words, just because it is a quote does not mean that it works inside qqww. I'm not sure if it is a bug or if is supposed to be this way.

[pmichaud]

Code: .say for qqww<foo „hello world” bar> # no quote protection here! Result: foo „hello world” bar

This looks like a Rakudo bug to me. Pm

[dha]

pmichaud - so, does that mean that that section of the grammar does represent what is supposed to be interpreted as a quote (even though, apparently, there's currently a bug)?

[pmichaud]

dha: I'm not sure I understand your question. Mainly, I would not want to see documentation that says "the definition of what is/is not a quote is in the grammar (lines ###-### of Grammar.nqp)", because Rakudo implements the language, it doesn't define it.

TL;DR of below: Rakudo's implementation of qqww seems bogus to me.

When I said "This looks like a Rakudo bug to me", it's because if low smart double-quotes are essentially synonymous to ASCII double-quoted strings in term context, then I would expect them to also act like ASCII double-quotes in a qqww context (i.e., provide quote protection).

Or, phrased slightly differently, S02 says "single angles do not interpolate, while double-angles do". Since double-angles are effectively the same as qq:ww:v/.../ and qqww says "with quote protection", I'd expect quote protection for any valid quote inside of the qqww, including «» and other quoting forms.

So, as a further example, I'm thinking Rakudo currently gets the following wrong, at least according to S02, since <<>> and «» are definitely "quotes".

> .say for << abc «def ghi» >>
abc
«def
ghi»
> .say for << abc <<def ghi>> >>
abc
<<
def
ghi
>>

At the very least the two above examples should be returning the same number of items (and to me, quoting delimiters shouldn't appear in the output).

Or, if "quote protection" is only meant for the string quoting delimiters, let's make that clear somewhere. But even here I would expect „hello world” to be quote-protected inside of qqww.

And finally a general observation: From the beginning it has been very very hard to properly implement qqww, «», and the like because they really don't fit neatly into standard grammar constructs. As the examples in this issue demonstrate, Rakudo still doesn't get a lot of cases correct (which is why one sometimes cannot use Rakudo to describe "what the language does").

Pm

[AlexDaniel]

RT #128304

[dha]

FWIW, I'm absolutely not suggesting we point people to the grammar in the docs. I am, however suggesting we might want to document what quote characters should be expected to work with respect to quote preservation (and possibly other contexts?)

That said, if, as in this case, rakudo implements the language in a way seemingly at odds with the spec, how should that be documented? I. e. in this situation, if curly quotes should be considered quote characters for :ww according to the language spec, but rakudo does not implement it that way... what do we tell users?

[pmichaud]

Wherever Rakudo deviates from what Perl 6 ought to be doing, I think we simply say "Rakudo's implementation is not entirely correct here" or something like that.

In this specific instance, perhaps something like: "Rakudo version 2016.xx has difficulty with quote protection inside of :ww; the only quote characters known to work are ..."

[AlexDaniel]

And perhaps a link to the bug report?

[coke]

This is a slippery slope. Rakudo is doing what the spec says it should be
doing (because it's passing the 6.c-errata spec tests every month before
release).

We don't want docs.perl6.org to be speculations; if there is an open ticket
with a commented out test in the errata branch for the current version of
the spec, then it might be reasonable to include - but we never went
through and removed all the todo'd and skip'd tests from the errata branch;
so there's been no vetting process to insure that those todo'd & skipped
tests are desired features.

I think this sort of thing has to be done on an ad hoc basis - that said, I
think this particular issue is safely clear of the slope and is safe to
include, as it looks like it will probably be added to the 6.c-errata when
it's fixed.

We can add documenting minimum versions of rakudo required to use a feature
as a new issue along with the to-be-done which-version-of-the-spec did this
feature exist in.

On Thu, Jun 2, 2016 at 11:56 PM, Patrick R. Michaud <
notifications@github.com> wrote:

Wherever Rakudo deviates from what Perl 6 ought to be doing, I think we
simply say "Rakudo's implementation is not entirely correct here" or
something like that.

In this specific instance, perhaps something like: "Rakudo version 2016.xx
has difficulty with quote protection inside of :ww; the only quotes known
to work are ..."

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
#533 (comment), or mute
the thread
https://github.com/notifications/unsubscribe/AADEUXa6TvqRzcojglV27cTl92J80YHuks5qH6XZgaJpZM4IjjJ4
.

Will "Coke" Coleda

[AlexDaniel]

Uh. That's a nice discussion, but the issue was fixed. Thanks everyone for your input.

There's a separate RT for quote protection issue. If it has to be discussed, then feel free to open another issue.