Dev-python/python-language-server: allow newer ujson versions #17882

AndrewAmmerlaan · 2020-10-11T11:18:56Z

This fixes a bug in pyls_jsonrpc preventing the use of ujson newer than 1.35 in pyls.

gentoo-bot · 2020-10-11T11:20:07Z

Pull Request assignment

Submitter: @AndrewAmmerlaan
Areas affected: ebuilds
Packages affected: dev-python/python-jsonrpc-server, dev-python/python-language-server

dev-python/python-jsonrpc-server: @AndrewAmmerlaan, @gentoo/proxy-maint
dev-python/python-language-server: @AndrewAmmerlaan, @gentoo/proxy-maint

Linked bugs

Bugs linked: 715290

In order to force reassignment and/or bug reference scan, please append [please reassign] to the pull request title.

Docs: Code of Conduct ● Copyright policy (expl.) ● Devmanual ● GitHub PRs ● Proxy-maint guide

gentoo-repo-qa-bot · 2020-10-11T12:01:48Z

Pull request CI report

Report generated at: 2020-10-11 12:01 UTC
Newest commit scanned: 2d97932
Status: ✅ good

There are existing issues already. Please look into the report to make sure none of them affect the packages in question:
https://qa-reports.gentoo.org/output/gentoo-ci/8ceda339ca/output.html

juippis

Could you revbump both of these so people who already have these installed can update to later ujson as well? git mv with your changes is fine here.

AndrewAmmerlaan · 2020-10-12T09:20:59Z

Could you revbump both of these so people who already have these installed can update to later ujson as well? git mv with your changes is fine here.

Done

gentoo-repo-qa-bot · 2020-10-12T09:36:51Z

Pull request CI report

Report generated at: 2020-10-12 09:36 UTC
Newest commit scanned: 87ab055
Status: ✅ good

There are existing issues already. Please look into the report to make sure none of them affect the packages in question:
https://qa-reports.gentoo.org/output/gentoo-ci/9d61aac212/output.html

juippis

Sooo after your update the tests fail with updated ujson,

============================================== FAILURES ==============================================
______________________________________ test_writer_bad_message _______________________________________

wfile = <_io.BytesIO object at 0x7f8541835e90>
writer = <pyls_jsonrpc.streams.JsonRpcStreamWriter object at 0x7f8541850210>

    def test_writer_bad_message(wfile, writer):
        # A datetime isn't serializable(or poorly serializable),
        # ensure the write method doesn't throw, but the result could be empty
        # or the correct datetime
        datetime.datetime = JsonDatetime
        writer.write(datetime.datetime(
            year=2019,
            month=1,
            day=1,
            hour=1,
            minute=1,
            second=1,
        ))
    
>       assert wfile.getvalue() in [
            b''
            b'Content-Length: 10\r\n'
            b'Content-Type: application/vscode-jsonrpc; charset=utf8\r\n'
            b'\r\n'
            b'1546300861'
            ]
E       AssertionError: assert b'Content-Length: 10\r\nContent-Type: application/vscode-jsonrpc; charset=utf8\r\n\r\n1546304461' in [b'Content-Length: 10\r\nContent-Type: application/vscode-jsonrpc; charset=utf8\r\n\r\n1546300861']
E        +  where b'Content-Length: 10\r\nContent-Type: application/vscode-jsonrpc; charset=utf8\r\n\r\n1546304461' = <built-in method getvalue of _io.BytesIO object at 0x7f8541835e90>()
E        +    where <built-in method getvalue of _io.BytesIO object at 0x7f8541835e90> = <_io.BytesIO object at 0x7f8541835e90>.getvalue

test/test_streams.py:124: AssertionError
====================================== short test summary info =======================================
FAILED test/test_streams.py::test_writer_bad_message - AssertionError: assert b'Content-Length: 10\...
==================================== 1 failed, 26 passed in 0.09s ====================================
 * ERROR: dev-python/python-jsonrpc-server-0.4.0-r1::testworld failed (test phase):
 *   Tests fail with python3.7

But the ::gentoo version passes all tests. So there's definitely still something broken with this.

AndrewAmmerlaan · 2020-10-12T14:06:17Z

Which ujson version where you using? this was working for me with ujson-4.0.0

juippis · 2020-10-12T14:09:35Z

[ebuild R ] dev-python/ujson-4.0.1::gentoo

Let me try with 4.0.0
Nope, it still fails :\
So it must be some missing dep issue.

Basically my deps are always the latest available version.

AndrewAmmerlaan · 2020-10-12T14:16:49Z

Basically my deps are always the latest available version.

Same, but I just retested and this works for me, so I'm a bit confused now

Lets compare equery depgraphs:

* dependency graph for dev-python/python-jsonrpc-server-0.4.0
`--  dev-python/python-jsonrpc-server-0.4.0  ~amd64 
`--  dev-python/ujson-4.0.0  (>=dev-python/ujson-3) ~amd64  [python_targets_python3_7(-)? python_targets_python3_8(-)? python_targets_python3_9(-)? -python_single_target_python3_7(-) -python_single_target_python3_8(-) -python_single_target_python3_9(-)]
`--  dev-lang/python-3.7.9  (dev-lang/python) ~amd64 
`--  dev-lang/python-3.8.6  (dev-lang/python) ~amd64 
`--  dev-lang/python-3.9.0  (dev-lang/python) ~amd64 
`--  dev-lang/python-exec-2.4.6-r2  (>=dev-lang/python-exec-2) ~amd64  [python_targets_python3_7(-)? python_targets_python3_8(-)? python_targets_python3_9(-)? -python_single_target_python3_7(-) -python_single_target_python3_8(-) -python_single_target_python3_9(-)]
`--  dev-python/versioneer-0.18-r1  (dev-python/versioneer) amd64  [python_targets_python3_7(-)? python_targets_python3_8(-)? python_targets_python3_9(-)? -python_single_target_python3_7(-) -python_single_target_python3_8(-) -python_single_target_python3_9(-)]
`--  dev-python/mock-3.0.5-r2  (dev-python/mock) amd64  [python_targets_python3_7(-)? python_targets_python3_8(-)? python_targets_python3_9(-)? -python_single_target_python3_7(-) -python_single_target_python3_8(-) -python_single_target_python3_9(-)]
`--  dev-python/pycodestyle-2.6.0  (dev-python/pycodestyle) amd64  [python_targets_python3_7(-)? python_targets_python3_8(-)? python_targets_python3_9(-)? -python_single_target_python3_7(-) -python_single_target_python3_8(-) -python_single_target_python3_9(-)]
`--  dev-python/pytest-6.1.1  (>=dev-python/pytest-4.5.0) ~amd64  [python_targets_python3_7(-)? python_targets_python3_8(-)? python_targets_python3_9(-)? -python_single_target_python3_7(-) -python_single_target_python3_8(-) -python_single_target_python3_9(-)]
`--  dev-python/setuptools-50.3.0  (>=dev-python/setuptools-42.0.2) ~amd64  [python_targets_python3_7(-)? python_targets_python3_8(-)? python_targets_python3_9(-)? -python_single_target_python3_7(-)-python_single_target_python3_8(-) -python_single_target_python3_9(-)]
[ dev-python/python-jsonrpc-server-0.4.0 stats: packages (11), max depth (1) ]

juippis · 2020-10-13T11:58:04Z

* dependency graph for dev-python/python-jsonrpc-server-0.4.0

Do note that your version is -r1, the ::gentoo version (-r0) works for me too as I said previously. But here's mine:

 * dependency graph for dev-python/python-jsonrpc-server-0.4.0-r1
 `--  dev-python/python-jsonrpc-server-0.4.0-r1  ~amd64 
   `--  dev-python/ujson-4.0.1  (>=dev-python/ujson-3) ~amd64  [python_targets_python3_7(-)? python_targets_python3_8(-)? python_targets_python3_9(-)? -python_single_target_python3_7(-) -python_single_target_python3_8(-) -python_single_target_python3_9(-)]
   `--  dev-lang/python-3.7.9  (dev-lang/python) ~amd64 
   `--  dev-lang/python-3.8.6  (dev-lang/python) ~amd64 
   `--  dev-lang/python-3.9.0  (dev-lang/python) ~amd64 
   `--  dev-lang/python-exec-2.4.6-r2  (>=dev-lang/python-exec-2) ~amd64  [python_targets_python3_7(-)? python_targets_python3_8(-)? python_targets_python3_9(-)? -python_single_target_python3_7(-) -python_single_target_python3_8(-) -python_single_target_python3_9(-)]
   `--  dev-python/versioneer-0.18-r1  (dev-python/versioneer) amd64  [python_targets_python3_7(-)? python_targets_python3_8(-)? python_targets_python3_9(-)? -python_single_target_python3_7(-) -python_single_target_python3_8(-) -python_single_target_python3_9(-)]
   `--  dev-python/mock-3.0.5-r2  (dev-python/mock) amd64  [python_targets_python3_7(-)? python_targets_python3_8(-)? python_targets_python3_9(-)? -python_single_target_python3_7(-) -python_single_target_python3_8(-) -python_single_target_python3_9(-)]
   `--  dev-python/pycodestyle-2.6.0  (dev-python/pycodestyle) amd64  [python_targets_python3_7(-)? python_targets_python3_8(-)? python_targets_python3_9(-)? -python_single_target_python3_7(-) -python_single_target_python3_8(-) -python_single_target_python3_9(-)]
   `--  dev-python/pytest-6.1.1  (>=dev-python/pytest-4.5.0) ~amd64  [python_targets_python3_7(-)? python_targets_python3_8(-)? python_targets_python3_9(-)? -python_single_target_python3_7(-) -python_single_target_python3_8(-) -python_single_target_python3_9(-)]
   `--  dev-python/setuptools-50.3.0  (>=dev-python/setuptools-42.0.2) ~amd64  [python_targets_python3_7(-)? python_targets_python3_8(-)? python_targets_python3_9(-)? -python_single_target_python3_7(-) -python_single_target_python3_8(-) -python_single_target_python3_9(-)]
[ dev-python/python-jsonrpc-server-0.4.0-r1 stats: packages (11), max depth (1) ]

AndrewAmmerlaan · 2020-10-13T15:30:09Z

Do note that your version is -r1, the ::gentoo version (-r0) works for me too as I said previously. But here's mine:

This is because the ebuild in my local overlay does not have the -r1 suffix.

Well I'm even more confused now, does it work for you without the patch? Because the error you're getting looks suspiciously like the patch.

juippis · 2020-10-13T16:05:31Z

Yep, they seem to pass happily without the patch.

AndrewAmmerlaan · 2020-10-13T18:18:03Z

Yep, they seem to pass happily without the patch.

The plot thickens, I get this without the patch:

=============================================================================================== FAILURES ================================================================================================
________________________________________________________________________________________ test_writer_bad_message ________________________________________________________________________________________

wfile = <_io.BytesIO object at 0x7efc2afa0bf0>, writer = <pyls_jsonrpc.streams.JsonRpcStreamWriter object at 0x7efc2afafd90>

def test_writer_bad_message(wfile, writer):
# A datetime isn't serializable(or poorly serializable),
# ensure the write method doesn't throw, but the result could be empty
# or the correct datetime
datetime.datetime = JsonDatetime
writer.write(datetime.datetime(
year=2019,
month=1,
day=1,
hour=1,
minute=1,
second=1,
))

>       assert wfile.getvalue() in [
b'',
b'Content-Length: 10\r\n'
b'Content-Type: application/vscode-jsonrpc; charset=utf8\r\n'
b'\r\n'
b'1546304461'
]
E       AssertionError: assert b'Content-Length: 10\r\nContent-Type: application/vscode-jsonrpc; charset=utf8\r\n\r\n1546300861' in [b'', b'Content-Length: 10\r\nContent-Type: application/vscode-jsonrpc; charset=utf8\r\n\r\n1546304461']
E        +  where b'Content-Length: 10\r\nContent-Type: application/vscode-jsonrpc; charset=utf8\r\n\r\n1546300861' = <built-in method getvalue of _io.BytesIO object at 0x7efc2afa0bf0>()
E        +    where <built-in method getvalue of _io.BytesIO object at 0x7efc2afa0bf0> = <_io.BytesIO object at 0x7efc2afa0bf0>.getvalue

test/test_streams.py:124: AssertionError

gentoo-repo-qa-bot · 2020-10-13T18:41:15Z

Pull request CI report

Report generated at: 2020-10-13 18:41 UTC
Newest commit scanned: 7ef354d
Status: ✅ good

There are existing issues already. Please look into the report to make sure none of them affect the packages in question:
https://qa-reports.gentoo.org/output/gentoo-ci/495ebfc611/output.html

juippis · 2020-10-14T08:16:37Z

I get the error you posted with the patch you included here. If no pathes are applied, python-jsonrpc-server-0.4.0-r1 passes all tests and installs just fine. Are you doing something weird on your end? :)

I'm applying this on top of ::gentoo tree and running emerge.

AndrewAmmerlaan · 2020-10-14T08:26:38Z

Are you doing something weird on your end? :)

Not that I know

Looking at the PR where I got this patch the upstream CI checks seem to be failing as well: palantir/python-jsonrpc-server#49

Though presumably the submitter wrote the patch in the first place because they were getting this error.

So it seems like some people need this patch to get the test to work, and for others the patch breaks the test.

I don't know what's going on, but the test seems to be doing something with date and time, so I suspect locale settings are causing these issues.

rossbridger · 2020-10-16T01:28:04Z

Probably upstream hasn't fully fixed this issue.

rossbridger · 2020-10-16T01:47:46Z

I took the liberty of fixing this issue, apply

diff --git a/test/test_streams.py b/test/test_streams.py
index 6985aec..9e64489 100644
--- a/test/test_streams.py
+++ b/test/test_streams.py
@@ -119,6 +119,7 @@ def test_writer_bad_message(wfile, writer):
         hour=1,
         minute=1,
         second=1,
+        tzinfo=datetime.timezone.utc
     ))

     assert wfile.getvalue() in [

It should work like a charm.

Closes: https://bugs.gentoo.org/715290 Package-Manager: Portage-3.0.8, Repoman-3.0.1 Signed-off-by: Andrew Ammerlaan <andrewammerlaan@riseup.net>

Bug: https://bugs.gentoo.org/715290 Package-Manager: Portage-3.0.8, Repoman-3.0.1 Signed-off-by: Andrew Ammerlaan <andrewammerlaan@riseup.net>

AndrewAmmerlaan · 2020-10-16T08:20:50Z

I took the liberty of fixing this issue, apply

diff --git a/test/test_streams.py b/test/test_streams.py
index 6985aec..9e64489 100644
--- a/test/test_streams.py
+++ b/test/test_streams.py
@@ -119,6 +119,7 @@ def test_writer_bad_message(wfile, writer):
         hour=1,
         minute=1,
         second=1,
+        tzinfo=datetime.timezone.utc
     ))

     assert wfile.getvalue() in [

It should work like a charm.

Awesome, Thank You!

This works for me, @juippis does this fix the issue for you too?

gentoo-repo-qa-bot · 2020-10-16T08:31:21Z

Pull request CI report

Report generated at: 2020-10-16 08:31 UTC
Newest commit scanned: 39ed72d
Status: ✅ good

There are existing issues already. Please look into the report to make sure none of them affect the packages in question:
https://qa-reports.gentoo.org/output/gentoo-ci/12c3b7cbde/output.html

juippis

Thanks, both of you!

gentoo-bot added self-maintained The PR changes only packages that are maintained by the submitter (i.e. no need to ask anybody else) assigned PR successfully assigned to the package maintainer(s). bug linked Bug/Closes found in footer, and cross-linked with the PR. labels Oct 11, 2020

juippis reviewed Oct 12, 2020

View reviewed changes

AndrewAmmerlaan force-pushed the dev-python/python-language-server branch from 2d97932 to 87ab055 Compare October 12, 2020 09:20

juippis reviewed Oct 12, 2020

View reviewed changes

AndrewAmmerlaan force-pushed the dev-python/python-language-server branch from 87ab055 to 7ef354d Compare October 13, 2020 18:21

rossbridger mentioned this pull request Oct 16, 2020

test_streams: Compare against string palantir/python-jsonrpc-server#49

Closed

AndrewAmmerlaan added 2 commits October 16, 2020 10:16

dev-python/python-jsonrpc-server: fix bug in tests with ujson>3

786ea6e

Closes: https://bugs.gentoo.org/715290 Package-Manager: Portage-3.0.8, Repoman-3.0.1 Signed-off-by: Andrew Ammerlaan <andrewammerlaan@riseup.net>

dev-python/python-language-server: allow newer ujson

39ed72d

Bug: https://bugs.gentoo.org/715290 Package-Manager: Portage-3.0.8, Repoman-3.0.1 Signed-off-by: Andrew Ammerlaan <andrewammerlaan@riseup.net>

AndrewAmmerlaan force-pushed the dev-python/python-language-server branch from 7ef354d to 39ed72d Compare October 16, 2020 08:19

AndrewAmmerlaan requested a review from juippis October 16, 2020 09:19

juippis approved these changes Oct 16, 2020

View reviewed changes

gentoo-bot closed this in 0e7bb9b Oct 16, 2020

AndrewAmmerlaan deleted the dev-python/python-language-server branch October 16, 2020 13:27

robbat2 added the hacktoberfest-accepted label Oct 30, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Dev-python/python-language-server: allow newer ujson versions #17882

Dev-python/python-language-server: allow newer ujson versions #17882

AndrewAmmerlaan commented Oct 11, 2020

gentoo-bot commented Oct 11, 2020

gentoo-repo-qa-bot commented Oct 11, 2020

juippis left a comment

AndrewAmmerlaan commented Oct 12, 2020

gentoo-repo-qa-bot commented Oct 12, 2020

juippis left a comment

AndrewAmmerlaan commented Oct 12, 2020

juippis commented Oct 12, 2020

AndrewAmmerlaan commented Oct 12, 2020

juippis commented Oct 13, 2020

AndrewAmmerlaan commented Oct 13, 2020

juippis commented Oct 13, 2020

AndrewAmmerlaan commented Oct 13, 2020

gentoo-repo-qa-bot commented Oct 13, 2020

juippis commented Oct 14, 2020

AndrewAmmerlaan commented Oct 14, 2020

rossbridger commented Oct 16, 2020

rossbridger commented Oct 16, 2020

AndrewAmmerlaan commented Oct 16, 2020

gentoo-repo-qa-bot commented Oct 16, 2020

juippis left a comment

Dev-python/python-language-server: allow newer ujson versions #17882

Dev-python/python-language-server: allow newer ujson versions #17882

Conversation

AndrewAmmerlaan commented Oct 11, 2020

gentoo-bot commented Oct 11, 2020

Pull Request assignment

Linked bugs

gentoo-repo-qa-bot commented Oct 11, 2020

Pull request CI report

juippis left a comment

Choose a reason for hiding this comment

AndrewAmmerlaan commented Oct 12, 2020

gentoo-repo-qa-bot commented Oct 12, 2020

Pull request CI report

juippis left a comment

Choose a reason for hiding this comment

AndrewAmmerlaan commented Oct 12, 2020

juippis commented Oct 12, 2020

AndrewAmmerlaan commented Oct 12, 2020

juippis commented Oct 13, 2020

AndrewAmmerlaan commented Oct 13, 2020

juippis commented Oct 13, 2020

AndrewAmmerlaan commented Oct 13, 2020

gentoo-repo-qa-bot commented Oct 13, 2020

Pull request CI report

juippis commented Oct 14, 2020

AndrewAmmerlaan commented Oct 14, 2020

rossbridger commented Oct 16, 2020

rossbridger commented Oct 16, 2020

AndrewAmmerlaan commented Oct 16, 2020

gentoo-repo-qa-bot commented Oct 16, 2020

Pull request CI report

juippis left a comment

Choose a reason for hiding this comment