jqlang release v1.7 breaks packtivity

[matthewfeickert]

jq (the language) had its first release in 5 years with jq v1.7, which includes some breaking changes.

jq the Python library added support for jqlang v1.7 in jq v1.6.0 and so v1.6.0 is also a breaking change for packtivity.

Action plan

• Add a cap on
  

  packtivity/setup.py

  Line 14 in fdeab4b

  "jq",

of <1.6.0

• c.f. build: Restrict jq to jqlang pre v1.7 #97

• Release packtivity v0.16.1 with this cap in it.
  • The jq capped packtivity will get picked up by the yadage ecosystem and subsequently keep other breaks from happening
  • c.f. https://github.com/yadage/packtivity/releases/tag/v0.16.1
• Find a way to fix packtivity to work with both jqlang v1.7 and before.
  • c.f. fix: Use 'paths(scalars)' as 'leaf_paths' filter is deprecated  #99
• Make a new packtivity v0.16.2 that adds this support and removes the cap.
  • c.f. https://github.com/yadage/packtivity/releases/tag/v0.16.2
• Yank packtivity v0.16.1 from PyPI.
  • c.f. https://pypi.org/project/packtivity/0.16.1/

[matthewfeickert]

cc @kratsg in the event that similar things hit any of your libraries.

[kratsg]

what actually breaks?

[matthewfeickert]

Here's an example failure for

packtivity/tests/test_asyncbackends.py

Lines 9 to 20 in 69aadb9

	def test_multiproc(tmpdir, basic_localfs_state, localproc_packspec):
	basic_localfs_state.ensure()
	pars = pdm.create({"outputfile": "{workdir}/helloworld.txt"})
	backend = MultiProcBackend(2)
	proxy = backend.submit(localproc_packspec, pars, basic_localfs_state)
	while not backend.ready(proxy):
	pass
	assert backend.successful(proxy)
	assert backend.result(proxy).json() == {
	"output": str(tmpdir.join("helloworld.txt"))
	}
	assert tmpdir.join("helloworld.txt").check() == True

=================================== FAILURES ===================================
________________________________ test_multiproc ________________________________

tmpdir = local('/tmp/pytest-of-runner/pytest-0/test_multiproc0')
basic_localfs_state = <LocalFSState rw: ['/tmp/pytest-of-runner/pytest-0/test_multiproc0'], ro: []>
localproc_packspec = {'environment': {'environment_type': 'localproc-env'}, 'process': {'cmd': 'echo Hello World > {outputfile}', 'process_type': 'string-interpolated-cmd'}, 'publisher': {'outputmap': {'output': 'outputfile'}, 'publisher_type': 'frompar-pub'}}

    def test_multiproc(tmpdir, basic_localfs_state, localproc_packspec):
        basic_localfs_state.ensure()
        pars = pdm.create({"outputfile": "{workdir}/helloworld.txt"})
        backend = MultiProcBackend(2)
        proxy = backend.submit(localproc_packspec, pars, basic_localfs_state)
        while not backend.ready(proxy):
            pass
>       assert backend.successful(proxy)
E       assert False
E        +  where False = <bound method MultiProcBackend.successful of <packtivity.asyncbackends.MultiProcBackend object at 0x7f68c3a[77](https://github.com/yadage/packtivity/actions/runs/6211558623/job/16860874815#step:6:78)160>>(<packtivity.asyncbackends.PacktivityProxyBase object at 0x7f68c3a1b4c0>)
E        +    where <bound method MultiProcBackend.successful of <packtivity.asyncbackends.MultiProcBackend object at 0x7f68c3a77160>> = <packtivity.asyncbackends.MultiProcBackend object at 0x7f68c3a77160>.successful

tests/test_asyncbackends.py:16: AssertionError
----------------------------- Captured stderr call -----------------------------
2023-09-17 04:27:26,0[86](https://github.com/yadage/packtivity/actions/runs/6211558623/job/16860874815#step:6:87) | pack.packtivity.step |   INFO | starting file logging for topic: step
_________________________________ test_maincli _________________________________

tmpdir = local('/tmp/pytest-of-runner/pytest-0/test_maincli0')

    def test_maincli(tmpdir):
        runner = CliRunner()
        result = runner.invoke(
            packtivity.cli.runcli,
            [
                "tests/testspecs/localtouchfile.yml",
                "-p",
                'outputfile="{workdir}/hello.txt"',
                "-w",
                str(tmpdir),
            ],
        )
>       assert result.exit_code == 0
E       AssertionError: assert 1 == 0
E        +  where 1 = <Result ValueError('jq: error: leaf_paths/0 is not defined at <top-level>, line 1:\nleaf_paths\njq: 1 compile error')>.exit_code

This kind of

jq: error: leaf_paths/0 is not defined at <top-level>, line 1:\nleaf_paths\njq: 1 compile error

is happening everywhere.

[matthewfeickert]

I think it might be something like jqlang/jq#38 (comment)

[matthewfeickert]

Ah, this would do it from https://github.com/jqlang/jq/releases/tag/jq-1.7:

Remove deprecated filters leaf_paths, recurse_down. @itchyny jqlang/jq#2666

given that leaf_paths is used in packtivity.

[matthewfeickert]

Closing as resolved with https://github.com/yadage/packtivity/releases/tag/v0.16.2