Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Python 3.11 compatibility #4401

Closed
5 tasks done
pekkaklarck opened this issue Jul 12, 2022 · 4 comments
Closed
5 tasks done

Python 3.11 compatibility #4401

pekkaklarck opened this issue Jul 12, 2022 · 4 comments
Labels
alpha 1 enhancement priority: high
Milestone
v6.0

Comments

@pekkaklarck
Copy link
Member

pekkaklarck commented Jul 12, 2022 •
edited

Python 3.11 is getting ready and we need to test that RF 5.1 is compatible with it. That requires:

  • Running all tests with the latest Python 3.11 version (beta 4 at the moment) both on Linux.
  • Same as above on Windows.
  • Fixing possible issues.
  • Adding Python 3.11 support to project metadata in setup.py.
  • Enabling Python 3.11 runs on CI (assuming it is already supported there).
@pekkaklarck pekkaklarck added this to the v5.1 milestone Jul 12, 2022
@pekkaklarck
Copy link
Member Author

pekkaklarck commented Jul 12, 2022 •
edited

Some tests failed. I analyzed why they failed and these were the reasons:

  1. Python 3.11 has enhanced error reporting and as a part of that tracebacks have ^^^^ markers showing where errors occur in more detail. This breaks some unit tests, but they are easy to fix by filtering such lines away from tracebacks before comparing them.
  2. typing.get_type_hints doesn't anymore automatically add None to the returned types if an argument has None as a default value. This means that if you have a keyword with a signature like arg: str = None, arg only has str as a type nowadays when earlier it had both str and None (i.e. NoneType). This means that if you call the keyword with a None object, it is not anymore accepted as-is but instead converted to a string None. I don't like out conversion behavior depending on Python versions, so I'll adapt the code so that None isn't converted if an argument has None as a default.
  3. Python 3.11 is more strict about embedding flags to regexp patterns. With earlier versions, for example, ^(?s)x works but it causes an error with Python 3.11. Luckily the only place in the code this error occurs isString.Get Lines Matching Regexp keyword and fixing it is easy.

pekkaklarck added a commit that referenced this issue Jul 12, 2022
@pekkaklarck
Stricter flag usage in regexp patterns.
e60897e 
Part of Python 3.11 compatibility. #4401
pekkaklarck added a commit that referenced this issue Jul 12, 2022
@pekkaklarck
Fix traceback related test on Python 3.11. #4401
b10ca90 
Filter away new `^^^` lines showing where the error occurred.
pekkaklarck added a commit that referenced this issue Jul 12, 2022
@pekkaklarck
Adjust code and tests to new typing.get_type_hints semantics.
e52c98f 
Python 3.11 changed `typing.get_type_hints` so that `None` isn't
anymore added to returned types automatically if an argument has
`None` as a default value. That changed our argument conversion logic
so that if you have an argument like `arg: str = None` and pass it a
`None` object, it isn't passed as-is but instead converted to a string
`None`. Argument conversion depending on Python versions wouldn't be
great, so this commit changes the logic so that `None` is never
converted if an argument has `None` as a default.

The same change also changed how types are shown by Libdoc. Earlier
`arg: str = None` was shown as `arg: str | None = None`, but nowadays
it's just `arg: str = None`. I prefer the new formatting and just
updated tests so that they expect that if Python 3.11+.

This is part of Python 3.11 compatibility. #4401
pekkaklarck added a commit that referenced this issue Jul 12, 2022
@pekkaklarck
Add Python 3.11 support to project metadata. #4401
b6e27e2
pekkaklarck added a commit that referenced this issue Jul 12, 2022
@pekkaklarck
Adjust test runs on CI.
6d02c2b 
- Run PRs against Python 3.6 and 3.10 instead of Python 3.6 and 3.9.
- Use Python 3.10 as the "runner side" Python with acceptance tests.
- Run tests also against Python 3.11. #4401
pekkaklarck added a commit that referenced this issue Jul 12, 2022
@pekkaklarck
New attempt to configure Python 3.11 run on CI. #4401
08e0758
pekkaklarck added a commit that referenced this issue Jul 12, 2022
@pekkaklarck
Try to fix atests failing on CI but not locally.
28cb9af 
These fail on CI with Python 3.11 due to `^^^` lines it adds to
tracebacks. It's strange same tests pass locally but hopefully
filtering those lines fixes tests on CI as well.

Part of #4401.
pekkaklarck added a commit that referenced this issue Jul 12, 2022
@pekkaklarck
Try to fix final atests failing on CI but not locally.
a685c99 
This test fails on CI with Python 3.11 due to `^^^` lines it adds to
tracebacks. It's strange it passes locally, but hopefully filtering
those lines fixes it on CI like it has fixed other similar tests.

Part of #4401.
pekkaklarck added a commit that referenced this issue Jul 12, 2022
@pekkaklarck
Don't run tests on CI with Python 3.11 on Windows. #4401
a417b5f 
Too many dependencies we need seem to be unavailable at the moment.
pekkaklarck added a commit that referenced this issue Jul 12, 2022
@pekkaklarck
New attempt to not run tests wilt Python 3.11 on Windows on CI. #4401
46fd8eb
@pekkaklarck
Copy link
Member Author

pekkaklarck commented Jul 12, 2022 •
edited

This is now done.

Getting tests to pass on CI turned out to be somewhat hard. Strangely more tests validating tracebacks failed on CI due to those ^^^ lines added by Python 3.11. CI uses beta 3 while I tested locally with beta 4, but it would be a strange if that affected this behavior. Anyway, I got tests passing also on CI by removing ^^^ lines.

Also needed to disable testing using Python 3.11 on Windows. Too many dependencies lacked Python 3.11 support.

@ShemTovYosef
Copy link

Hi @pekkaklarck

We want to move to python 3.11 and are currently blocked due to this issue: robotframework/PythonRemoteServer#78
Do you have any plans for it?

@pekkaklarck
Copy link
Member Author

PythonRemoteServer has been neglected for too long. Now that RF 6.0 is out, I ought to have time to look at it.

@pekkaklarck pekkaklarck mentioned this issue Mar 26, 2024
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
alpha 1 enhancement priority: high
Projects
None yet
Milestone
v6.0
Development

No branches or pull requests
2 participants
@pekkaklarck @ShemTovYosef