Python 3.8 #142
Python 3.8 #142
Conversation
See pygments/lexers/python.py in Github, commit a8a1a3a06e364bc8f5e777bf663e9da4710cb055
In 3.6.7 the tokenize module now implicitly emits a NEWLINE token when provided with input that does not have a trailing new line. This behavior now matches what the C tokenizer does internally. (Contributed by Ammar Askar in bpo-33899.) "rinoh --help" command now works.
who knows what it should be.
DeprecationWarning: invalid escape sequence \
DeprecationWarning: invalid escape sequence \
From https://docs.python.org/3/whatsnew/3.7.html : PEP 479 is enabled for all code in Python 3.7, meaning that StopIteration exceptions raised directly or indirectly in coroutines and generators are transformed into RuntimeError exceptions. (Contributed by Yury Selivanov in bpo-32670.)
RefKeyDictionary moved to its own file. This will be done for other classes, functions, etc., and unit tests will be done for each.
See PEP 479.
…ue (inheriting from StopIteration).
"Removed __str__ implementations from builtin types bool, int, float, complex and few classes from the standard library. They now inherit __str__() from object. As result, defining the __repr__() method in the subclass of these classes will affect their string representation. (Contributed by Serhiy Storchaka in bpo-36793.)" https://bugs.python.org/issue36793
|
|
|
Thank you. This is much appreciated! Please accept my apologies for not responding to your offer for help in #133. I still had a bunch of local commits sitting on my hard drive, some of which already fixed a few of these issues. This week I finally reserved some time for rinohtype and I have now pushed these to GitHub. I will go through your commits and merge them into the master branch on Friday. |
|
A small update. I didn't get too far yet, as I first wanted to fix the Travis CI configuration and make sure all builds succeed. |
|
No problem. Thanks for letting me know.
…-Norm
On Fri, 24 Jan 2020 at 09:31, Brecht Machiels ***@***.***> wrote:
A small update. I didn't get too far yet, as I first wanted to fix the
Travis CI configuration and make sure all builds succeed.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
<#142?email_source=notifications&email_token=ABV73HKFHRUQKGDJUWO5PQTQ7MJUXA5CNFSM4KKG77WKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEJ3K3DQ#issuecomment-578203022>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ABV73HNA45FBV7I2D4YH5ATQ7MJUXANCNFSM4KKG77WA>
.
|
| @@ -145,7 +145,7 @@ def __new__(cls, value, base=10, indirect=False): | |||
| def __init__(self, value, base=10, indirect=False): | |||
| Object.__init__(self, indirect) | |||
|
|
|||
| def _repr(self): | |||
| def __repr__(self): | |||
There was a problem hiding this comment.
Maybe just __repr__ = int.__repr__?
| @@ -561,7 +561,7 @@ def parse_string(cls, string): | |||
|
|
|||
| @classmethod | |||
| def doc_repr(cls, value): | |||
| space = '\ ``\N{NO-BREAK SPACE}``\ ' | |||
| space = r'\ ``\N{NO-BREAK SPACE}``\ ' | |||
There was a problem hiding this comment.
It does not work because of \N.
| space = r'\ ``\N{NO-BREAK SPACE}``\ ' | |
| space = '\\ ``\N{NO-BREAK SPACE}``\\ ' |
| inserted between each two consecutive elements""" | ||
| iterable = iter(iterable) | ||
| yield next(iterable) | ||
| while True: |
There was a problem hiding this comment.
It can be simpler:
for item in iterable:
yield element
yield item|
|
||
| def __init__(self, *args, **kwargs): | ||
| self.store = dict() | ||
| self.update(dict(*args, **kwargs)) |
There was a problem hiding this comment.
Why not just self.update(*args, **kwargs)?
Actually, it does not work with kwargs at all, because you create a weak reference to a key, but kwargs keys are strings, which do not support weak references.
There was a problem hiding this comment.
Because I copied this code from StackOverflow without thinking :-)
| del self.store[id(obj)] | ||
|
|
||
| def __iter__(self): | ||
| return iter(self.store) |
There was a problem hiding this comment.
Note that it emits ids of keys, not keys. So many code (for example values() and items()) will not work.
| return value | ||
|
|
||
| def __setitem__(self, obj, value): | ||
| self.store[id(obj)] = ref(obj), value |
There was a problem hiding this comment.
You do not keep a hard reference to obj. It can be removed, and after that its id is no longer valid. New created object can take the same id.
| else: | ||
| char = chr(code) | ||
| if char in ('\\', '(', ')'): | ||
| char = '\\' + char | ||
| if char in (r'\\', r'(', r')'): |
There was a problem hiding this comment.
r'\\' is a two-character string. char is a single-character string, it never can be equal to r'\\'.
I think the existing code (without r) is correct.
| if char in ('\\', '(', ')'): | ||
| char = '\\' + char | ||
| if char in (r'\\', r'(', r')'): | ||
| char = r'\\' + char |
There was a problem hiding this comment.
Have doubts about this change.
|
Thanks for pulling in some of these changes. Feel free to delete this PR. Don't worry about the unit tests and other changes; these were more for my curiosity. Master seems to work fine now on 3.8. |
For now, I've cherry picked only those that were needed to get the the build on Travis and AppVeyor in a more or less OK state. Fixing the CI configuration took up a lot of time, unfortunately. I'm currently looking into fixing the long-running tests. Once that is done, I'll get back to cherry-picking your commits. I definitely welcome more unit tests! Some notes so far: |
|
Brecht,
-
- No worries about the VSCode files; I've wondered what to do with
them, since I like to track them, but don't want to put in PRs.
- Hadn't hear of pyenv. I'll check it out, thanks. So many tools!
Also...
- Any changes left over from the PR I'll resubmit as individual changes,
simpler to deal with.
- Btw, do you have any notes or documentation on the architecture of
rinohtype? Your code is excellent, but sometimes I find it difficult to
follow what the code is doing.
Thanks,
…-Norm Lorrain
On Wed, 29 Jan 2020 at 10:48, Brecht Machiels ***@***.***> wrote:
Thanks for pulling in some of these changes. Feel free to delete this PR.
Don't worry about the unit tests and other changes; these were more for my
curiosity. Master seems to work fine now on 3.8.
For now, I've cherry picked only those that were needed to get the the
build on Travis and AppVeyor in a more or less OK state. Fixing the CI
configuration took up a lot of time, unfortunately. I'm currently looking
into fixing the long-running tests. Once that is done, I'll get back to
cherry-picking your commits. I definitely welcome more unit tests!
Some notes so far:
- I've opted not to include the VSCode-related files. I don't think we
should store these in the repository.
- I'm using pyenv <https://github.com/pyenv/pyenv> for managing
different versions of CPython. I see that there's also a Windows version
pyenv-win <https://github.com/pyenv-win/pyenv-win> nowadays. This
works well together with tox. I'll try to describe this setup.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
<#142?email_source=notifications&email_token=ABV73HMGE27AF7BS7L7RXV3RAG6PTA5CNFSM4KKG77WKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEKIDW2Y#issuecomment-579877739>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ABV73HIET7R4ALNIF3DZZGDRAG6PTANCNFSM4KKG77WA>
.
|
Too many tools, indeed! I wrote some documentation on the tox environments and pyenv in
There's no need for that. I'm happy cherry-picking the commits, which I will now get back to. After that, I hope to make a long-overdue new release.
Yes, the program flow can get very complex. I tend to get lost myself too! One cause of this is the gratuitous use of exceptions during the rendering. This is something I'd like to address (#130). Unfortunately, there is no documentation on the architecture. I'm sure writing it would uncover several more opportunities for simplification. Alas, I don't think I will find the time to do that any time soon. But if you have any questions (specific or general) with regard to the architecture, don't hesitate to ask me! |
|
I went over the remaining commits in this PR and cherry-picked some of them onto master. Some remarks:
Thanks for your contributions. Sorry this took so long; I again ended up wrestling with Travis CI configuration which is a major time drain (required dependencies for PDF diffing). Next up: a new release. Also, thank you @serhiy-storchaka for your input! I fixed the issue in RefKeyDictionary which you spotted. |
This PR brings Python 3.8 compatibility. Many changes here, so to make things easier on myself I created virtual environments for Python 3.5, 3.6, 3.7, 3.8, and identified what need to be changed at each step. This PR contains the results. Tested under Windows and Windows Subsystem for Linux (Debian).
Also added some unit tests, for two reasons:
I'd like to see more tests in the future. To make this easier, begun to refactor util.py to a package. Feel free to omit that change. This is a work in progress.
The changes to Python 3.7 affecting the project are:
The changes to Python 3.8 affecting the project are:
I also addressed code that emitted warnings (e.g. DeprecationWarning: invalid escape sequence \)