-
Notifications
You must be signed in to change notification settings - Fork 373
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Replace apply
with unpacking operators
#1325
Conversation
Oh great, Travis is showing that it segfaults on 3.3 and 3.4 for some reason. Probably because PEP 448 only made it into 3.5. I'll gate those tests appropriately and see what happens. |
Nope. I guess this won't be so easy. There were probably changes to Python's expectations for the AST. But please do take a look at this before I start hacking away at that. |
Having written three of those four issues, I approve of the additional syntax. However, I didn't say anything about removing To answer my own question, perhaps in macros and higher-order functions? Would we ever need the old One other issue I take with the proposed implementation is how you lex the tokens. A tag macro can be any symbol, yet the tests demonstrate I think whitespace should be required between a |
Yes,
Not if you want to be able to call it. Here are two examples:
So the effect of this change is that a tag macro's name can't begin with an asterisk, the same way it can't begin with |
Beyond testing Hy itself, why would you construct a literal list or dictionary only to unpack it? |
I'm saying it should be valid syntactically, not that I had a use case in mind. But think about how this could interact with the threading macros In the case of a dictionary, even in Python, you can use dictionary unpacking to put the kwarg name in a variable. >>> def bar(*, a=None, b=None):
print(a,b)
>>> foo = 'a'
>>> bar(**{foo: 1})
1 None
>>> foo = 'b'
>>> bar(**{foo: 1})
None 1 |
Oh, that's a good point. |
I said any symbol, not any string. I meant the kind of symbol you can type in directly, without going through a We do have a |
Okay, here's another counterexample:
|
Yikes! I'd call that one that a bug in the lexer. I think we were only using that for the shebang on the first line? We should be able to use |
So, to be clear, you want |
It seems much more consistent to me to implement |
(deftag * [form] `(unpack-iterable ~form))
(deftag ** [form] `(unpack-mapping ~form)) Then you don't need the special case in the compiler, and don't have to change the lexer. You still need the new special forms.
No, |
All right, that makes sense. |
Okay, Pythons 3.3 and 3.4 should work now. However, I learned the hard way that the |
It had better, or I don't think we can do it. This setup is causing errors in macros.
That is not supposed to happen. Quasiquote is broken. Let's try that manually.
Macros are supposed to recursively expand before special forms are applied. What's going on here? |
You get this error because Note that it would do you no good for me to add
That isn't quite true. A macro is only expanded once |
I did the manual build wrong. Let's try an explicit Hy model.
That worked. Breaking quasiquote is not okay though. I know some macros in Common Lisp will recursively macroexpand their body before acting on it. We might need that feature to implement this properly, unless we can come up with a better idea. |
Hey, this works too.
Are you sure we can't implement it as a tag macro? Which part is failing? |
Not in general. The case you tried doesn't require the compiler to see the
The compiler sees
It's not broken. Quasiquote can't do what you want it to do here because of Hy's basic strategy for compilation. The most that can be done is to produce a more informative error message.
That's out of scope for this PR.
Yes, nor with a regular macro.
Any part that requires looking at child forms. Check where I've used the function |
Also, I pushed https://github.com/Kodiologist/hy/tree/unpacking-tag-macro if you want to experiment with my attempt. |
More weirdness.
Why does the Python translation have a |
I got that far by treating the |
I noticed that, too. I think it's a bug (or rather, a missing feature) in |
But not 2.7? @berkerpeksag are we doing something wrong with astor to get these None keys? I thought we weren't supporting 2.6 anymore? |
Sorry, that was a typo for 3.5 and 3.6. Fixed. |
To answer the obvious question, in earlier Pythons, dictionary unpacking is represented by passing the |
Examples work better.
I think I get it now. The |
One of the problem cases was unpacking a mapping into a dict display. I've been playing around with this a bit. It's possible to set an arbitrary attr flag in the ast itself. This seems to work in Python 3.6. @builds(HyDict)
def compile_dict(self, m):
keyvalues, ret, _ = self._compile_collect(m)
temp = []
for kv in keyvalues:
if hasattr(kv, "unpack_mapping"):
temp.extend((None, kv))
else:
temp.append(kv)
keyvalues = temp
ret += ast.Dict(lineno=m.start_line,
col_offset=m.start_column,
keys=keyvalues[::2],
values=keyvalues[1::2])
return ret
@builds("unpack_mapping")
@checkargs(exact=1)
def compile_unpack_mapping(self, expr):
ret = self.compile(expr[1])
ret.expr.unpack_mapping = True
return ret Older ast versions will have to build it differently, but it seems like it should work. |
Well done. But, making it work for all uses of the unpacking operator will require rejiggering the order in which child forms are compiled in several places in the compiler. Where the compiler currently inspects the HyModels of child forms and compiles them after, you'll have to compile them first and then inspect the Python AST. As an analogy, the following doesn't work
because the compiler looks for a literal HyKeyword when it's compiling In all, it seems like a lot of work for the mere bragging rights of being able to say that we implemented |
I feel like that would work in other Lisps. Hy makes an unusual distinction between quoted and unquoted keywords, but only in function calls. So that's like calling This is a wart in Hy, but I don't know that we can fix it syntactically to make it always quoted, given how Python works. (Given The fact that that macro doesn't work bothers me, but with unpacking, I could mostly work around it. This might be something to fix in the compiler later.
I'm not clear on which parts aren't working. Where are the operators used in Python? Unpack for mappings is used in
Was that all of them? That's one we can implement as a tag macro, at least. Unpack for iterables is used in
Was that all of them?
I don't see it as mere bragging rights. First, it would keep those special cases out of the lexer, which should be kept as simple as is reasonable. A DSL could use Second, and more importantly, I want them to work properly as special forms even without the tags. The new special forms are not well-behaved now. They break quasiquote, and probably break in macros in ways a user would find hard to predict. If they worked consistently, they could be (tag) macros. |
Yes, that's everything I implemented.
Redefining a core macro seems like a really bad idea. Python lets you assign something to
So we'll cross that bridge when we come to it.
But they are well-behaved and work consistently with the way the rest of the compiler works. For example, look at all the instances in If you want to change the compiler in deep ways, I'm certainly open to such a PR, but again, it's out of scope for this PR. |
Ha. Now which of us is illiberal? (1246). Yes, this could absolutely be abused and one should be careful. But see the |
All right, this is ready for review now. |
(FWIW I wonder if we can use the same syntax ideas to implement tuple unpacking such...) |
and such |
But tuple unpacking is among what's implemented here. Or did you mean something other than Python's one-star unpacking? |
I guesss I forgot to `pip install --upgrade` last night; `apply` went away in hylang/hy#1325.
It's possible to provoke a wide variety of cryptic error messages by misusing
#*
and#**
, but I'm not sure it's worth trying to beautify all those possible errors in this pull request.