Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Canonicalize case of TAP query parameter id #357

Merged
merged 1 commit into from
Sep 17, 2022
Merged

Canonicalize case of TAP query parameter id #357

merged 1 commit into from
Sep 17, 2022

Conversation

rra
Copy link
Contributor

@rra rra commented Sep 16, 2022

When retrieving the contents of a TAP query from an AsyncTAPJob, lowercase the identifier of the parameter before comparing it to "query". Some TAP servers reflect the case of the parameter in the job XML and DALI says that parameter keys are case-insensitive, so the parameter ID may be "QUERY" or some other case.

test_submit_job was using the wrong arguments to submit_job() (passing a URL first instead of the query), uncovered by testing this change. Fix that as well.

In MockAsyncTAPServer, stop lowercasing incoming parameters and instead reflect the provided case in the parameter ID, making it possible to test this fix.

Fixes #356

@codecov
Copy link

codecov bot commented Sep 16, 2022

Codecov Report

Merging #357 (044b433) into main (861298f) will increase coverage by 0.07%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##             main     #357      +/-   ##
==========================================
+ Coverage   78.27%   78.35%   +0.07%     
==========================================
  Files          46       46              
  Lines        5497     5497              
==========================================
+ Hits         4303     4307       +4     
+ Misses       1194     1190       -4     
Impacted Files Coverage Δ
pyvo/dal/tap.py 70.62% <100.00%> (+1.12%) ⬆️

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

@bsipocz bsipocz added the bug label Sep 16, 2022
@bsipocz
Copy link
Member

bsipocz commented Sep 16, 2022

Could you please add a one liner to the CHANGELOG.rst? Thanks!

@bsipocz bsipocz added this to the v1.4 milestone Sep 16, 2022
@@ -175,11 +175,11 @@ def parameters(self, request, context):
if 'QUERY' in data:
assert data['QUERY'] == 'SELECT TOP 42 * FROM ivoa.obsCore'
for param in job.parameters:
if param.id_ == 'query':
if param.id_.lower() == 'query':
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure any of these should be handled in the tests itself, instead, add an example that uses different cases, so we in fact test for the fact that the code handles it in a case-insensitive way.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The changes to the MockAsyncTAPServer have to be done to prevent it from lowercasing all incoming job parameters, thus making it impossible to test this fix. Once it no longer lowercases parameter ids, the other places where it references parameter ids also have to canonicalize. I didn't make that very clear in the PR, now fixed.

Copy link
Member

@bsipocz bsipocz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will need a changelog, and some work on the test case.

When retrieving the contents of a TAP query from an AsyncTAPJob,
lowercase the identifier of the parameter before comparing it to
"query". Some TAP servers reflect the case of the parameter in the
job XML and DALI says that parameter keys are case-insensitive, so
the parameter ID may be "QUERY" or some other case.

test_submit_job was using the wrong arguments to submit_job()
(passing a URL first instead of the query), uncovered by testing
this change.  Fix that as well.

In MockAsyncTAPServer, stop lowercasing incoming parameters and
instead reflect the provided case in the parameter ID, making it
possible to test this fix.

Fixes #356
@rra
Copy link
Contributor Author

rra commented Sep 17, 2022

CHANGES.rst entry added. I took the liberty of adding it to the changelog for 1.3.1 rather than 1.4, in the hope that this is a sufficiently small bug fix that it could be pulled to the release branch.

If one disables the test suite's mock server's lowercasing of parameter names, the existing tests will already test this if one adds an assert with the query accessor, but I added a test that explicitly uses a mixed-case parameter name just to make the testing clearer (and added a comment about what's going on here).

@rra
Copy link
Contributor Author

rra commented Sep 17, 2022

Oh, hm, I made an assumption about why there are separate 1.3.1 and 1.4 entries in CHANGES.rst that appears to be wrong; there isn't a separate 1.3 branch that I see. I'm now confused about where to put the changes entry; let me know if I put it in the wrong place.

@bsipocz
Copy link
Member

bsipocz commented Sep 17, 2022

CHANGES.rst entry added. I took the liberty of adding it to the changelog for 1.3.1 rather than 1.4, in the hope that this is a sufficiently small bug fix that it could be pulled to the release branch.

Thanks. At this point I don't think there will be a 1.3.1, but we'll go straight to 1.4 but that is something we'll fix in the release process.

Copy link
Member

@bsipocz bsipocz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks all good, thanks!

@bsipocz bsipocz merged commit c7cb3eb into astropy:main Sep 17, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

AsyncTAPJob's "query" attribute is not robust to case variation among community TAP servers
2 participants