Preserve string kind modifiers

[ilevkivskyi]

This will help experimenting with python/mypy#3648

Currently we preserve only b string modifier on Python 2, this is a bit arbitrary and for the above mentioned PR we need to keep u. Instead of adding another special-casing I just always preserve all string modifiers in a short string kind on Str node, for example:

>>> st = ast3.parse("u'hi'")
>>> st.body[0].value.kind
'u'

[JelleZijlstra]

Does typed-ast have backwards compatibility guarantees? You're dropping the has_b attribute from ast27 Str nodes.

[ilevkivskyi]

@JelleZijlstra FWIW I added it some time ago by request from @gvanrossum but it was never used since then. It stayed deprecated for few last releases: https://github.com/python/typed_ast/blob/master/typed_ast/ast27.py#L23

[ilevkivskyi]

Moreover, the new kind attribute can still be used instead if we will return to b later (only "API" is changed, not functionality).

[gvanrossum]

Thanks! Using my new powers as a typed_ast committer to merge this.

[ddfisher]

👍

All differences from the standard Python AST are documented in the module documentation. @ilevkivskyi, would you mind updating the docs here and here?

[ilevkivskyi]

All differences from the standard Python AST are documented in the module documentation. @ilevkivskyi, would you mind updating the docs here and here?

Yes, sure, I will make a PR with the docs. But probably not this week.