Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix compatibility with Jython #378

Merged
merged 1 commit into from Jan 13, 2021
Merged

Fix compatibility with Jython #378

merged 1 commit into from Jan 13, 2021

Conversation

@anishathalye
Copy link
Contributor

@anishathalye anishathalye commented Jan 22, 2020

This patch was taken from #369 (comment), authored by Pekka Klärck peke@iki.fi.

In short, Jython doesn't support lone surrogates, so importing yaml (and in particular, loading reader.py) caused a UnicodeDecodeError. This patch works around this through a clever use of eval to defer evaluation of the string containing the lone surrogates, only doing it on non-Jython platforms.

This is only done in lib/yaml/reader.py and not lib3/yaml/reader.py because Jython does not support Python 3.

With this patch, Jython's behavior with respect to Unicode code points over 0xFFFF becomes as it was before 0716ae2. It still does not pass all the unit tests on Jython (passes 1275, fails 3, errors on 1); all the failing tests are related to unicode. Still, this is better than simply crashing upon import yaml.

With this patch, all tests continue to pass on Python 2 / Python 3.

CC @pekkaklarck

Fixes #369

This patch was taken from
#369 (comment),
authored by Pekka Klärck <peke@iki.fi>.

In short, Jython doesn't support lone surrogates, so importing yaml (and
in particular, loading `reader.py`) caused a UnicodeDecodeError. This
patch works around this through a clever use of `eval` to defer
evaluation of the string containing the lone surrogates, only doing it
on non-Jython platforms.

This is only done in `lib/yaml/reader.py` and not `lib3/yaml/reader.py`
because Jython does not support Python 3.

With this patch, Jython's behavior with respect to Unicode code points
over 0xFFFF becomes as it was before
0716ae2. It still does not pass all the
unit tests on Jython (passes 1275, fails 3, errors on 1); all the
failing tests are related to unicode. Still, this is better than simply
crashing upon `import yaml`.

With this patch, all tests continue to pass on Python 2 / Python 3.
@anishathalye anishathalye requested a review from perlpunk Jan 22, 2020
@pekkaklarck
Copy link

@pekkaklarck pekkaklarck commented Jan 23, 2020

Looks great to me! The reasoning is very well explained in the description above. Thanks for creating the PR @anishathalye and hopefully someone can merge it!

@perlpunk
Copy link
Member

@perlpunk perlpunk commented May 24, 2020

@ingydotnet any news on release planning?

@ingydotnet ingydotnet changed the base branch from master to release/5.4 Jan 13, 2021
@nitzmahone nitzmahone moved this from To do to Done in PyYAML 5.4 release planning Jan 13, 2021
@ingydotnet ingydotnet merged commit a60f7a1 into yaml:release/5.4 Jan 13, 2021
15 checks passed
asherf added a commit to asherf/pants that referenced this issue Jan 21, 2021
5.4.1 (2021-01-20)

* yaml/pyyaml#480 -- Fix stub compat with older pyyaml versions that may unwittingly load it

5.4 (2021-01-19)

* yaml/pyyaml#407 -- Build modernization, remove distutils, fix metadata, build wheels, CI to GHA
* yaml/pyyaml#472 -- Fix for CVE-2020-14343, moves arbitrary python tags to UnsafeLoader
* yaml/pyyaml#441 -- Fix memory leak in implicit resolver setup
* yaml/pyyaml#392 -- Fix py2 copy support for timezone objects
* yaml/pyyaml#378 -- Fix compatibility with Jython

https://github.com/yaml/pyyaml/blob/master/CHANGES
stuhood pushed a commit to pantsbuild/pants that referenced this issue Jan 21, 2021
5.4.1 (2021-01-20)

* yaml/pyyaml#480 -- Fix stub compat with older pyyaml versions that may unwittingly load it

5.4 (2021-01-19)

* yaml/pyyaml#407 -- Build modernization, remove distutils, fix metadata, build wheels, CI to GHA
* yaml/pyyaml#472 -- Fix for CVE-2020-14343, moves arbitrary python tags to UnsafeLoader
* yaml/pyyaml#441 -- Fix memory leak in implicit resolver setup
* yaml/pyyaml#392 -- Fix py2 copy support for timezone objects
* yaml/pyyaml#378 -- Fix compatibility with Jython

https://github.com/yaml/pyyaml/blob/master/CHANGES
otc-zuul bot pushed a commit to opentelekomcloud-infra/customer-service-monitoring that referenced this issue Mar 29, 2021
Bump pyyaml from 5.3.1 to 5.4 in /roles/prepare_build/files

Bumps pyyaml from 5.3.1 to 5.4.

Changelog
Sourced from pyyaml's changelog.

5.4 (2021-01-19)

yaml/pyyaml#407 -- Build modernization, remove distutils, fix metadata, build wheels, CI to GHA
yaml/pyyaml#472 -- Fix for CVE-2020-14343, moves arbitrary python tags to UnsafeLoader
yaml/pyyaml#441 -- Fix memory leak in implicit resolver setup
yaml/pyyaml#392 -- Fix py2 copy support for timezone objects
yaml/pyyaml#378 -- Fix compatibility with Jython




Commits

58d0cb7 5.4 release
a60f7a1 Fix compatibility with Jython
ee98abd Run CI on PR base branch changes
ddf2033 constructor.timezone: _copy & deepcopy
fc914d5 Avoid repeatedly appending to yaml_implicit_resolvers
a001f27 Fix for CVE-2020-14343
fe15062 Add 3.9 to appveyor file for completeness sake
1e1c7fb Add a newline character to end of pyproject.toml
0b6b7d6 Start sentences and phrases for capital letters
c976915 Shell code improvements
Additional commits viewable in compare view




Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR:

@dependabot rebase will rebase this PR
@dependabot recreate will recreate this PR, overwriting any edits that have been made to it
@dependabot merge will merge this PR after your CI passes on it
@dependabot squash and merge will squash and merge this PR after your CI passes on it
@dependabot cancel merge will cancel a previously requested merge and block automerging
@dependabot reopen will reopen this PR if it is closed
@dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
@dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
@dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
@dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
@dependabot use these labels will set the current labels as the default for future PRs for this repo and language
@dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
@dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
@dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

You can disable automated security fix PRs for this repo from the Security Alerts page.

Reviewed-by: None <None>
Reviewed-by: Anton Kachurin <katchuring@gmail.com>
Reviewed-by: Anton Sidelnikov <None>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
No open projects
Linked issues

Successfully merging this pull request may close these issues.

5 participants