JMESPath Extractor - in progress under marklz #78
Comments
|
JMESPath may be a better option for some use cases, and supports returning the last element in an array, for example. |
|
Sam - is it something and a junior-midlevel python coder can help you with? |
|
@marklz Absolutely! It probably will not require much code to do the coupling, and all the aspects needed have documentation and examples (JSONschema extension, extensions.md for how to write an extension, and resttest.py loading of the jsonschema extension). Fairly easy to throw together tests for this as well. |
|
looked briefly at JMESPath ( https://github.com/jmespath/jmespath.py and the spec in the link above) - is it much more complex than this? I'm surely missing something... import jmespath
class JMESPathExtractor(AbstractExtractor):
""" Extractor that uses JMESPath syntax
See http://jmespath.org/specification.html for details
"""
extractor_type = 'jmespath'
is_body_extractor = True
def extract_internal(self, query=None, args=None, body=None, headers=None):
try:
body = json.loads(body)
return jmespath.search(query, body)
except ValueError:
raise ValueError("Not legal JSON!")
@classmethod
def parse(cls, config):
base = JMESPathExtractor()
return cls.configure_base(config, base)
return base |
|
@marklz Nope, not much more complex than that for the core! There's a couple more pieces to get it PR-ready, but not big ones. First we need some basic unit tests (and I'm not sure here if JMESPath for python returns JSON text fragments or Python objects, but if the first one, we need to call the json.loads function to parse them so the comparators work). Second, is an autoload for the extension like we have for the jsonschema validator. So, less than a half-dozen lines of code, and there's already an example to work from. Third: an entry in the advanced_guide with the registered name & description, a syntax example, and a link to the JMESPath page (for more advanced uses). Finally: I do apologize for the delay! I've been sick with flu the last few days and only just got well enough to start looking at code again. This looks like it would be a great addition to PyrestTest, and thank you! |
|
No worry about reply times - I have a full time job too :-)
Option 2 - create a extractor_jmespath.py in ext/ folder, and at the end of that file add - not clear what. jsonschema has and then in resttest.py add So, which way should I go?
Mark. |
|
Ah, those full-time jobs! Unit tests: I think that what you describe is more work than you really need. Really we only need to exercise config parsing, basic query use, templating in the query, and error handling (including empty value returns). As a nice-to-have, it would be helpful to have a test in a ComparatorValidator. Autoload: Option 2, please! This way it's possible to successfully run PyRestTest without the Jmespath library installed, but if the library is present the extension is available. And yes, the example you gave really is all you need there (the extension loader looks for registry variables) -- both snippets look correct to me at first glance. On consideration, it would be nice to convert the extension_use_test.sh to some form of functional test for autoloading extensions (that only runs if libraries are present)... but I'm certainly not making that a PR requirement since it's something I ought to have done a while back. Documentation: Ha! It doesn't even need that much, just an example of syntax for using the extractor and what that returns, and a link to the tutorial there. Hope that makes your life easier as you have a fulltime job too! ;-) |
|
Can't register extension - what am I doing wrong? |
|
@marklz Simple, if it's using the --import_extensions argument, the path should be pyresttest.ext.extractor_jmespath, I believe -- it's relative to current folder. If you're doing import like the jsonschema extractor, it's of course going to omit the first bit. |
|
Good news - autoloader works all is well; I was running pyresttest - which was /usr/bin/pyrestest! we started to pick up correct code. Without any modifications, run my initial test - first two tests run correctly, throws exception on 3rd one. I'll debug it tomorrow, but it's a nice progress. |
|
@marklz Excellent! |
|
I'm going over http://jmespath.org/tutorial.html and it looks good so far:
I assume that for real unit tests, I'll have to convert URL requests to reading files - how do I handle from packaging POV? Put input files into ext? Into some other place? Mark. |
and get no output. When I run against URL, all is well - I get what am I doing wrong? Mark. |
|
@marklz This is looking really great! For your real unit tests, we really only need to test that:
As far as structure for tests goes: for functional tests (pyresttest/functionaltests.py), add a conditional to the functional tests to run if JMESPath can import, i.e.: try:
import JMESPath
def run_jmespath_test(self):
do_my_testing_here
catch ImportError:
pass # Doesn't run JMESPath test if can't import libraryFor unit tests, we need a test class in pyresttest/ext/test_jmespath.py. Documentation: Running without URL: |
|
I fixed cut down on documentation, but I'm unclear on where to put unit tests. I see two options:
# Handle url/test as named, or, failing that, positional arguments
if not args['url'] or not args['test']:
if len(unparsed_args) == 2:
args[u'url'] = unparsed_args[0]
args[u'test'] = unparsed_args[1]
elif len(unparsed_args) == 1 and args['url']:
args['test'] = unparsed_args[0]
elif len(unparsed_args) == 1 and args['test']:
args['url'] = unparsed_args[0]
else:
parser.print_help()
parser.error(
"wrong number of arguments, need both url and test filename, either as 1st and 2nd parameters or via --url and --test")we have only length(unparsed_args) == 1, unparsed_args[0] is ext/jmespath-test.yaml, it's assumed to be url, which is latter happily opened and processed... Mark. |
Exactly! And also something like test_parse_validator_extracttest.
That's the snippet I posted earlier, with the imports. Because imports don't have to be at the top of python files and can be executed conditionally, like I'd listed earlier try:
import JMESPath
def run_jmespath_test(self):
do_my_testing_here
catch ImportError:
pass # Doesn't run JMESPath test if can't import libraryExcept do_my_testing_here is something similar to: https://github.com/svanoort/pyresttest/blob/master/pyresttest/functionaltest.py#L111 At least, I think this should work, I have not actually tested it yet! Also pay no mind to how horrifyingly awful some of the other test syntax is -- eventually I need to rework the internal Python APIs to be somewhat more elegant (probably mimicking frisby.js a bit), but that's going to be a future-roadmap thing, around the PyRestTest 2.0 release probably. ;-)
D'oh! |
svanoort
changed the title
svanoort
modified the milestones:
1.7.0 - Python 3 + Parsing/Configuration Internals,
Misc Enhancements
|
I somehow manage to fail in both tests:
def test_parse_validator_jmespath_extracttest(self):
""" Test parsing for jmespath extract test """
config = {
'jmespath': 'key.val',
'test': 'exists'
}
myjson_pass = '{"id": 3, "key": {"val": 3}}'
myjson_fail = '{"id": 3, "key": {"valley": "wide"}}'
validator = validators.ExtractTestValidator.parse(config)and get How do I force loading of jmespath extractor?
|
|
For the test_validators unit test, the problem is very simple: resttest.py isn't loaded by the validators module, so the extension is never loaded. Resttest can't be imported there, since resttest imports validators, and if they load each other we get an endless loop (this is why the functional test exists, to exercise the auto-import). The solution is simple: for the extension test, we'll need to explicitly do the try-import section (in case the library isn't installed) and then at the beginning of the test call the validators.register_extractor (after importing the extension): https://github.com/marklz/pyresttest/blob/master/pyresttest/validators.py#L542 Be warned, it may be a little tricky to get the import right for this! For the functional test:
|
|
@marklz ^ |
|
Good news - test_validators unit test is done
def test_get_validators_jmespath_fail(self):
""" Test validators that should fail """
test = Test()
test.url = self.prefix + '/api/person/'
test.validators = list()
print 'jmespath validators: ' + str(test.validators)
cfg_exists = {'jmespath': 'meta.limit', 'test': 'exists'}
test.validators.append(
validators.parse_validator('extract_test', cfg_exists))
test_response = resttest.run_test(test)
print 'jmespath response: ' + str(test_response)result of execution (I've added spaces for readability): |
|
@marklz Wanna jump on gitter and discuss? I'm at a local python project night and now would be perfect timing! :-) |
|
@marklz Specifically, https://gitter.im/svanoort/pyresttest |
|
@marklz I've got the fix (tested with a clone of your code): This: https://github.com/marklz/pyresttest/blob/master/pyresttest/ext/extractor_jmespath.py#L30 Should instead be: Ast.eval and json.loads are not interchangeable (the first loads a python object, the second loads json). The two syntaxes are very similar but not interchangeable. |
|
@marklz It's now merged in; whew, let it never be said that dual python 2/3 compatibility is easy! Had to make some changes to get everything to play nicely now. |
What:
As a pyresttest user who needs to do more detailed JSON analysis, I would like to be able to use full jsonpath support in working with request bodies (extraction for variables and content analysis). This should be available if the library is installed and fail if not.
How:
I would like to add a jsonpath-rw extractor, set up to auto-register in the same way as the jsonschema validator.
It will also require adding extending the autoload extensions to iterate through a list of extensions. For now, that's sufficient but it might be desirable to try to autoload all of the 'ext' folder content.
This is an isolated task, easy enough for someone to execute as a PR.
The text was updated successfully, but these errors were encountered: