PyRestTest provides hooks for extending built-in components with your own Python code.
- In general: use more advanced dependencies while not making them required for installation
- Generators: generate data for templating URL/request body/tests/etc
- Extractors: get data from HTTP response body/headers and use it in future tests
- Validators: write custom tests using headers & response bodies
- Test Functions: for the ExtractTest validator, validate a single condition
- Comparator function:s for the ComparatorValidator, compare expected and actual values
Extensions are specified for loadin at runtime with the --import_extensions argument:
python pyresttest/resttest.py https://api.github.com extension_use_test.yaml --import_extensions 'sample_extension'Extensions are python module names, separated by semicolons:
python pyresttest/resttest.py https://api.github.com fancypants_test.yaml --import_extensions 'fancy_validator;form_data_generator'import pyresttest.validators as validators
# Define a simple generator that doubles with each value
def parse_generator_doubling(config):
start = 1
if 'start' in config:
start = int(config['start'])
# We cannot simply use start as the variable, because of scoping limitations
def generator():
val = start
while(True):
yield val
val = val*2
return generator()
GENERATORS = {'doubling': parse_generator_doubling}If this is imported when executing the test, you can now use this generator in tests.
See the sample extension.
It shows an extension for all extensible functions.
- Function(s) to run
- Registry Entries: these are special ALLCAPS variables binding extension names
These are the simplest, one-argument functions that return True or False
def test(x):
return x in [1, 2, 3]Compare two values and return True/False
def compare(a, b):
return a > bThese are standard python generators. There is ONE twist, they should be infinite (for benchmark use).
The function takes one argument, config, which is a string or dictionary of arguments for creating the generator.
def factory_choice_generator(values):
""" Return a generator that picks values from a list randomly """
def choice_generator():
my_list = list(values)
rand = random.Random()
while(True):
yield random.choice(my_list)
return choice_generator
def parse_choice_generator(config):
""" Parse choice generator """
vals = config['values']
if not vals:
raise ValueError('Values for choice sequence must exist')
if not isinstance(vals,list):
raise ValueError('Values must be a list of entries')
return factory_choice_generator(vals)()The function for the registry would be parse_choice_generator
These need to be objects, and should extend pyresttest.AbstractExtractor The 'parse' function below will be registered in the registry.
Example:
class HeaderExtractor(AbstractExtractor):
""" Extractor that pulls out a named header """
extractor_type = 'header' # Printable name for the type
is_header_extractor = True # Use headers in extraction
is_body_extractor = False # Uses body in extraction
def extract_internal(self, query=None, args=None, body=None, headers=None):
""" The real logic, extract a value, using a templated query string and args
The query is an attribute stored in the parent, and templating is used
"""
try:
return headers[query]
except Exception:
return None
@classmethod
def parse(cls, config, extractor_base=None):
base = HeaderExtractor()
# Base parser automatically handles templating logic
# And reads the query
return cls.configure_base(config, base)Validators should extend AbstractValidator. The parse function below will be registered in the registry VALIDATORS.
class ExtractTestValidator(AbstractValidator):
""" Does extract and test from request body """
name = 'ExtractTestValidator'
extractor = None
test_fn = None
test_name = None
config = None
@staticmethod
def parse(config):
""" Config is a dict """
output = ExtractTestValidator()
config = parsing.lowercase_keys(parsing.flatten_dictionaries(config))
output.config = config
extractor = _get_extractor(config)
output.extractor = extractor
test_name = config['test']
output.test_name = test_name
test_fn = VALIDATOR_TESTS[test_name]
output.test_fn = test_fn
return output
def validate(self, body=None, headers=None, context=None):
try:
extracted = self.extractor.extract(body=body, headers=headers, context=context)
except Exception as e:
return Failure(message="Exception thrown while running extraction from body", details=e, validator=self)
tested = self.test_fn(extracted)
if tested:
return True
else:
failure = Failure(details=self.config, validator=self)
failure.message = "Extract and test validator failed on test: {0}({1})".format(self.test_name, extracted)
return failureThe extension loader will look for special registry variables in the module and attempt to load them.
Registries are dictionarys of {registered_name: function}. Registry names are ALWAYS case-insensitive, since they are keywords for the YAML syntax.
These are:
- VALIDATOR_TESTS - function is just thetest function
- COMPARATORS - function is just the comparison function
- GENERATORS - function is a parse function to get a generator
- EXTRACTORS - function is a parse function returning an AbstractExtractor implementation
- VALIDATORS - function is a parse function returning an AbstractValidator implementation
Each one maps to the same registry in pyresttest.validators.
- Need to generate complex, formatted data?
- Write a generator extension, or multiple generators may be used together to create a complex result
- Want to test whether API results fit a business rule?
- Write a validator extension, your logic can be as complex as you like
- Want to apply a business rule to the output and use the result?
- Write an extractor extension
- You can do testing with the result via the validators ExtractTest and ComparatorValidator
- By declaring the extractor with the test, it can be used in future tests
- Want to test with complex matching logic between two parts of a response?
- Write a Comparator to do the comparison, use extractors for pulling out the components
- Want to run external logic after a test?
- Want to interact with an external system (a DB, for example) before tests?
- Write a custom generator function returning data
- Want to confirm results were written to a database?
- Write a custom validator or extractor that speaks to the database
- An extractor can return a value from the DB for comparison
- A validator can do the database fetch and return a failure if it was not right