salt.utils.http.query results not consistent across backends.

[damon-atkins]

Description of Issue/Question

salt.utils.http.query result ouput not consistent across backends.

The following is inside salt.utils.http.query

    if not backend:
        backend = opts.get('backend', 'tornado')

salt.utils.http.query returns a dict. This dict always contains handle. However handle can be:

• requests.models.Response from requests.Session()
• tornado.httpclient.HTTPResponse from tornado

It appears backend set to urllib2 may not work.

salt.utils.http.query should return a consistent result. e.g.

{
response_headers: [list in order received]
response_status: <User friendly message>
body: <response>
size: <int>
Optionally consider some of the standard response hearders like Etag, Last-Modified, Data, Content-Type, Content-Length, Expires, Host, Cache-Control
}

Setup

Steps to Reproduce Issue

Change backend and run the test suite. Or use fileclient.get_url()

Versions Report

Development, 2018.X 2017.X

[Ch3LL]

seems we could improve the returns here.

Can you describe what problems this is causing for you with the different returns?

[damon-atkins]

fileclient.get_url gets from salt.utils.http.query the following

tornado comes back with

{
'body': b'',
'handle': HTTPResponse(_body=b'',buffer=<_io.BytesIO object at 0x000001FBA95D4A98>,code=200,effective_url='https://nmap.org/dist/nmap-7.01-setup.exe',error=None,headers=<tornado.httputil.HTTPHeaders object at 0x000001FBA9737390>,reason='OK',request=<tornado.httpclient.HTTPRequest object at 0x000001FBA95CBA20>,request_time=25.246567964553833,time_info={})
}

With urllib2 these lines fail

        if stream is True or handle is True:
            return {
                'handle': result,
                'body': result.content,  # AttributeError: 'HTTPResponse' object has no attribute 'content'
            }

request comes back with
{'handle': <class 'requests.models.Response'>}

If someone changes the minion.conf backend option, things will most likely break.

[damon-atkins]

I was looking into how get_url() works and hence salt.utils.http.query() because salt.utils.http.query currently does not return response headers and hence get_url can not use response headers.

[Ch3LL]

@damon-atkins thanks for clarifying the issues :)

ping @saltstack/team-core any ideas here?

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

If this issue is closed prematurely, please leave a comment and we will gladly reopen the issue.

[damon-atkins]

Pls reopen, this is not resolve yet. The PR was closed above, without being implemented.

[stale]

Thank you for updating this issue. It is no longer marked as stale.