Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update test suites and exclusion list #239

Closed

Conversation

itchyny
Copy link
Contributor

@itchyny itchyny commented Nov 24, 2022

This PR updates the test suites from the jq repository, and also updates the exclusion list.

@itchyny itchyny force-pushed the update-test-suites-20221124 branch 2 times, most recently from 0c5dfe7 to 8d788c9 Compare November 29, 2022 12:06
@eiiches
Copy link
Owner

eiiches commented Dec 4, 2022

Thanks for improving tests!

Can you exclude this test case for now? The test tries to compare the output of stedolan/jq and jackson-jq (if jq cli is available on the machine), but stedolan/jq 1.6 has a bug that causes the test to time out.

- q: 'index("")'
  in: ""
  out:
  - null
  v: '[1.6, 1.6]'

Or, preferably, you can change the download script like below to mark the index("") test as v1.7, so that the test will not be executed on stedolan/jq 1.6.

emit_tests('[1.5, 1.5]', load_jq_tests_from_url('https://raw.githubusercontent.com/stedolan/jq/jq-1.5/tests/jq.test'), 'tests/jq-1.5.yaml')
emit_tests('[1.5, 1.5]', load_jq_tests_from_url('https://raw.githubusercontent.com/stedolan/jq/jq-1.5/tests/onig.test'), 'tests/jq-1.5-onig.yaml')
emit_tests('[1.5, 1.5]', load_jq_manual('https://raw.githubusercontent.com/stedolan/jq/master/docs/content/manual/v1.5/manual.yml'), 'tests/jq-1.5-manual.yaml')

emit_tests('[1.6, 1.6]', load_jq_tests_from_url('https://raw.githubusercontent.com/stedolan/jq/jq-1.6/tests/jq.test'), 'tests/jq-1.6.yaml')
emit_tests('[1.6, 1.6]', load_jq_tests_from_url('https://raw.githubusercontent.com/stedolan/jq/jq-1.6/tests/onig.test'), 'tests/jq-1.6-onig.yaml')
emit_tests('[1.6, 1.6]', load_jq_manual('https://raw.githubusercontent.com/stedolan/jq/master/docs/content/manual/v1.6/manual.yml'), 'tests/jq-1.6-manual.yaml')

emit_tests('[1.7, 1.7]', load_jq_tests_from_url('https://raw.githubusercontent.com/stedolan/jq/master/tests/jq.test'), 'tests/jq-1.7.yaml')
emit_tests('[1.7, 1.7]', load_jq_tests_from_url('https://raw.githubusercontent.com/stedolan/jq/master/tests/onig.test'), 'tests/jq-1.7-onig.yaml')
emit_tests('[1.7, 1.7]', load_jq_manual('https://raw.githubusercontent.com/stedolan/jq/master/docs/content/manual/manual.yml'), 'tests/jq-1.7-manual.yaml')

@itchyny
Copy link
Contributor Author

itchyny commented Dec 18, 2022

Hi, I updated the download scripts to save the tests from the master branch as jq-latest-*.yaml, and assert the test cases assuming that jq command (with no version suffix) to be the latest version. We cannot tell what version the next release will be, but we can mark them as (1.6, ] as for now. What do you think?

@itchyny itchyny closed this Aug 1, 2023
@itchyny itchyny deleted the update-test-suites-20221124 branch August 1, 2023 23:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants