The pyramid_input
package is a pyramid plugin that adds a "tween" that parses and combines all data presented in the HTTP request in one standardized request attribute. The following data is currently accepted (in increasing order of precedence):
- Query string parameters
- Request payload (i.e. the "request body") in several different formats
See Example for a detailed example.
- Homepage: https://github.com/canaryhealth/pyramid_input
- Bugs: https://github.com/canaryhealth/pyramid_input/issues
$ pip install pyramid_input
Enable the tween either in your INI file via:
pyramid.includes = pyramid_input
or in code in your package's application initialization via:
def main(global_config, **settings):
# ...
config.include('pyramid_input')
# ...
If needed, adjust pyramid_input's behaviour by setting the various Configuration options in your INI file.
Then, access a request's input parameters, regardless of their origin or encoding, simply as:
def request_handler(request):
if request.input.somekey == 'some-value':
...
The pyramid_input tween makes all of the following requests present the identical structure in the request.input attribute:
Interpreting GET data:
GET /path?foo=bar&zig.zag=zog&zig.zen-0=mig&zig.zen-1=mag
Interpreting and merging GET and POST form data:
POST /path?foo=bar
Content-Type: application/x-www-form-urlencoded
zig.zag=zog&zig.zen-0=mig&zig.zen-1=mag
Interpreting and merging GET and JSON payload data:
POST /path?foo=bar
Content-Type: application/json
{"zig": {"zag": "zog", "zen": ["mig", "mag"]}}
Interpreting and merging GET and YAML payload data:
POST /path?foo=bar
Content-Type: application/yaml
zig:
zag: zog
zen:
- mig
- mag
Interpreting and merging GET and XML payload data:
POST /path?foo=bar
Content-Type: application/xml
<zig>
<zag>zog</zag>
<zen>mig</zen>
<zen>mag</zen>
</zig>
All of the above will result in the identical data structure being contained in the request.input attribute:
request.input = {
'foo': 'bar',
'zig': {
'zag': 'zog',
'zen': ['mig', 'mag']
}
}
Please note that in all cases the original parameters (in request.GET, request.POST, request.params, request.body and request.json_body) are left as-is, so if the raw data is needed, it can be accessed directly.
The HTTP query string parameters are "unflattened" using FormEncode's variable_decode implementation, which converts the key-value pairs into a nested tree structure. For example, the following query string parameters:
?simple=val1&dict.subkey=val2&dict.list-0=el0&dict.list-1=el1
is transformed into the structure:
{
simple: val1
dict: {
subkey: val2
list: [ el0, el1 ]
}
}
Note that query string parameters are by default overwritten by any HTTP request payload data.
The request payload (aka. request body) is parsed when the request's Content-Type
is one of the following values:
application/x-www-form-urlencoded
,multipart/form-data
The standard HTTP POST encoding; the key-value pairs are parsed exactly the same way as the query string, i.e. they are unflattened using FormEncode's variable_decode implementation. Note that for
multipart/form-data
(the content-type used for standard HTTP-based file uploading), nothing special is done: the file object that is in request.POST is simply referenced as-is in request.input as well.application/json
,application/x-json
,text/json
,text/x-json
,...+json
The payload is parsed using the built-in JSON parser, and no further processing is done. The data is required to be a dictionary unless the pyramid_input.require-dict is set to false, and if this is violated, request processing is aborted with a 400 "Bad Request" response error.
application/yaml
,application/x-yaml
,text/yaml
,text/x-yaml
,...+yaml
If the PyYAML package is available, the payload is parsed using the YAML parser, and no further processing is done. The data is required to be a dictionary unless the pyramid_input.require-dict is set to false, and if this is violated, request processing is aborted with a 400 "Bad Request" response error.
application/xml
,application/x-xml
,text/xml
,text/x-xml
,...+xml
The payload is parsed using the built-in ElementTree parser and the XML document is converted to a tree via a fairly simplistic mapping process. Note that this mapping process is "lossy", i.e. some aspects of the XML serialization (such as order of interleaved non-similar children nodes) are lost in the convertion.
When both query string paramaters and a payload is specified in the request, the output of parsing both data sources are then combined together to form a single data tree. By default (i.e. when pyramid_input.combine.deep is true), the payload data overrides the query string data by overlaying and merging the two tree structures recursively.
A recursive deep merge basically means that dict keys get merged with dict keys, and all other type combinations turn into lists.
The deep merge can be disabled by setting the pyramid_input.combine.deep option to false, in which case the payload top-level dict keys completely override the query string top-level keys, without any inspection of sub-keys.
For example, given the following query string tree structure:
foo: bar
dict:
list: [a, b]
item: c
and the following payload tree structure:
bar: foo
dict:
list: d
item: e
a deep merge will result in:
foo: bar
bar: foo
dict:
list: [a, b, d]
item: [c, e]
and a non-deep merge will result in:
foo: bar
bar: foo
dict:
list: d
item: e
The following configuration settings can be set in your application's main
section:
pyramid_input.enabled
: bool, default: truepyramid_input.attribute-name
: str, default: 'input'pyramid_input.combine.deep
: bool, default: truepyramid_input.include
: list(glob), default: /**pyramid_input.exclude
: list(glob), default: nullpyramid_input.require-dict
: bool, default: truepyramid_input.fail-unknown
: bool, default: trueIf the request's Content-Type is unknown (or its parser disabled) and pyramid_input.fail-unknown is true, a 400 "bad request" error is returned. If set to false, the payload is ignored.
pyramid_input.native-dict
: bool, default: falseIf true, request.input will be a standard python dict object. If false (the default), then it will be a recursive aadict object (which is a subclass of dict that supports attribute-based access to its items).
pyramid_input.error-handler
: symbol-spec, default: nullOn error (such as a 400 "Bad Request" for invalid JSON), this function is called with the pyramid.httpexceptions.Exception subclass that caused the error. The default implementation has this function signature equivalent:
def function(request, error): return error
pyramid_input.reparse-methods
: list(str), default: 'PATCH'Enable a workaround for pyramid 1.4.2+ that does not properly parse the application/x-www-form-urlencoded request body if the request method is PATCH. It is unknown if other methods have this issue or if it has been fixed.
pyramid_input.json.enable
: bool, default: truepyramid_input.json.parser
: symbol-spec, default: nullSpecifies the JSON parser. If not specified, uses Pyramid's pre-existing
Request.json_body
attribute.pyramid_input.yaml.enable
: bool, default: truepyramid_input.yaml.parser
: symbol-spec, default: 'yaml.load'pyramid_input.xml.enable
: bool, default: truepyramid_input.xml.parser
: symbol-spec, default: 'xml.etree.ElementTree.fromstring'