Request is fired to packages.httpie.io on every invocation of `http` #1527

thenbe · 2023-09-04T15:10:54Z

A request is fired to packages.httpie.io on every invocation of http. Not sure if this related to update notifications (#1136). If so, then it seems overkill to check for updates on every single invocation.

You can see and/or block this request using something like opensnitch (linux) or littlesnitch (mac).

Checklist

I've searched for similar issues.
I'm using the latest version of HTTPie.

Minimal reproduction code and steps

Run http example.com
Use opensnitch/littlesnitch (links above) to view request to packages.httpie.io
Again, run http example.com
Again, use opensnitch/littlesnitch (links above) to view request to packages.httpie.io
repeat...

Current result

Unnecessary/excessive requests to packages.httpie.io.
…

Expected result

No unnecessary/excessive requests to packages.httpie.io.
…

Debug output

Please re-run the command with --debug, then copy the entire command & output and paste both below:

$ http --debug example.com
HTTPie 3.2.2
Requests 2.31.0
Pygments 2.15.1
Python 3.10.12 (main, Jun  6 2023, 22:43:10) [GCC 12.3.0]
/nix/store/pkj7cgmz66assy7l18zc7j992npb41nx-python3-3.10.12/bin/python3.10
Linux 6.1.49

<Environment {'apply_warnings_filter': <function Environment.apply_warnings_filter at 0x7f1fdd949510>,
 'args': Namespace(),
 'as_silent': <function Environment.as_silent at 0x7f1fdd9493f0>,
 'colors': 256,
 'config': {'default_options': []},
 'config_dir': PosixPath('/home/nbe/.config/httpie'),
 'devnull': <property object at 0x7f1fdd92b880>,
 'is_windows': False,
 'log_error': <function Environment.log_error at 0x7f1fdd949480>,
 'program_name': 'http',
 'quiet': 0,
 'rich_console': <functools.cached_property object at 0x7f1fdd935780>,
 'rich_error_console': <functools.cached_property object at 0x7f1fdd9373d0>,
 'show_displays': True,
 'stderr': <_io.TextIOWrapper name='<stderr>' mode='w' encoding='utf-8'>,
 'stderr_isatty': True,
 'stdin': <_io.TextIOWrapper name='<stdin>' mode='r' encoding='utf-8'>,
 'stdin_encoding': 'utf-8',
 'stdin_isatty': True,
 'stdout': <_io.TextIOWrapper name='<stdout>' mode='w' encoding='utf-8'>,
 'stdout_encoding': 'utf-8',
 'stdout_isatty': True}>

<PluginManager {'adapters': [],
 'auth': [<class 'httpie.plugins.builtin.BasicAuthPlugin'>,
          <class 'httpie.plugins.builtin.DigestAuthPlugin'>,
          <class 'httpie.plugins.builtin.BearerAuthPlugin'>],
 'converters': [],
 'formatters': [<class 'httpie.output.formatters.headers.HeadersFormatter'>,
                <class 'httpie.output.formatters.json.JSONFormatter'>,
                <class 'httpie.output.formatters.xml.XMLFormatter'>,
                <class 'httpie.output.formatters.colors.ColorFormatter'>]}>

>>> requests.request(**{'auth': None,
 'data': RequestJSONDataDict(),
 'headers': <HTTPHeadersDict('User-Agent': b'HTTPie/3.2.2')>,
 'method': 'get',
 'params': <generator object MultiValueOrderedDict.items at 0x7f1fdd4033e0>,
 'url': 'http://example.com'})

HTTP/1.1 200 OK
Accept-Ranges: bytes
Age: 597962
Cache-Control: max-age=604800
Content-Encoding: gzip
Content-Length: 648
Content-Type: text/html; charset=UTF-8
Date: Mon, 04 Sep 2023 15:08:27 GMT
Etag: "3147526947"
Expires: Mon, 11 Sep 2023 15:08:27 GMT
Last-Modified: Thu, 17 Oct 2019 07:18:26 GMT
Server: ECS (dcb/7F83)
Vary: Accept-Encoding
X-Cache: HIT

<!doctype html>
<html>
<head>
    <title>Example Domain</title>

    <meta charset="utf-8" />
    <meta http-equiv="Content-type" content="text/html; charset=utf-8" />
    <meta name="viewport" content="width=device-width, initial-scale=1" />
    <style type="text/css">
    body {
        background-color: #f0f0f2;
        margin: 0;
        padding: 0;
        font-family: -apple-system, system-ui, BlinkMacSystemFont, "Segoe UI", "Open Sans", "Helvetica Neue", Helvetica, Aria
l, sans-serif;
        
    }
    div {
        width: 600px;
        margin: 5em auto;
        padding: 2em;
        background-color: #fdfdff;
        border-radius: 0.5em;
        box-shadow: 2px 3px 7px 2px rgba(0,0,0,0.02);
    }
    a:link, a:visited {
        color: #38488f;
        text-decoration: none;
    }
    @media (max-width: 700px) {
        div {
            margin: 0 auto;
            width: auto;
        }
    }
    </style>    
</head>

<body>
<div>
    <h1>Example Domain</h1>
    <p>This domain is for use in illustrative examples in documents. You may use this
    domain in literature without prior coordination or asking for permission.</p>
    <p><a href="https://www.iana.org/domains/example">More information...</a></p>
</div>
</body>
</html>

Additional information, screenshots, or code examples

…

The text was updated successfully, but these errors were encountered:

jkbrzt · 2023-09-08T09:48:25Z

Thanks for the report. This is supposed to happen infrequently. Will fix.

tgrushka · 2024-02-25T07:07:53Z

Still happening. Very annoying testing a proxy server getting an extra request on every request.

OmichronAgain · 2024-03-30T17:04:26Z

Still happening on the latest version (3.2.2).
While having a periodic update check may be useful, perhaps there could be an option in the config file to disable it entirely?
Either way, the additional request on every invocation makes it significantly harder to use for testing a proxy setup.
Any solution would be appreciated.

Ousret · 2024-04-04T06:00:08Z

I tried to reproduce but without success. Can, everyone in this thread, post the debug output so that we can establish the common factor?

thenbe · 2024-04-04T07:32:10Z

I can reproduce this still on 3.2.2. @Ousret To help you reproduce this, can you share what method you're using to detect the request? (I mentioned a couple of example methods in my original post)

OmichronAgain · 2024-04-04T11:00:09Z

@Ousret running mitmproxy listening on port 8080.

Running HTTPS_PROXY=http://localhost:8080 https --verify=no get example.com once results in

Running the same command twice results in

No matter what kind of request I make with HTTPie, it always fires off a check to packages.httpie.io every time.

Getting the debug output:

Hopefully this is useful.

close issue httpie#1527

Ousret · 2024-04-04T14:04:28Z

Found the origin of this situation. A fix is available in #1531
To preview it, run the following pip install "git+https://github.com/Ousret/HTTPie.git@feature-tryout-niquests" -U

Ousret · 2024-04-04T17:17:50Z

Can you test the patch and report back? If it solved your issue or not.

OmichronAgain · 2024-04-04T18:56:35Z

@Ousret well, it's certainly different.

It seems that HTTPie is still issuing the requests, but now doesn't trust the proxy's cert (even with --verify=no).

close issue httpie#1527

Ousret · 2024-04-05T04:52:15Z

HTTPie initially set verif=False (v3.x) for fetching latest versions information from the server, the given PR let it (cert verif) by default. If you don't use mitmproxy and the request succeed, it will stop doing it.
Nonetheless, I applied an additional measure that (i) consider a failure sufficient to stop trying (ii) forward the verify argument to it.

You can try to install the patch again, it should be OK now.

close issue httpie#1527

OmichronAgain · 2024-04-05T09:36:15Z

Yup, seems to work!

I tested various orderings of using proxy/not using proxy (in a Docker container so different tests did not influence each other) and it no longer does the update check every time. Thank you for the hard work!

Now just have to wait for #1531 to land...

thenbe added bug Something isn't working new Needs triage. Comments are welcome! labels Sep 4, 2023

jkbrzt self-assigned this Sep 8, 2023

Ousret added a commit to Ousret/httpie that referenced this issue Apr 4, 2024

fix infinite loop update checker in a specific condition

fb54d2b

close issue httpie#1527

Ousret linked a pull request Apr 4, 2024 that will close this issue

Proposal v4.0 b1 (HTTP/2+3, OS Trust Store, Custom DNS, OCSP, ...) #1531

Open

Ousret added a commit to Ousret/httpie that referenced this issue Apr 5, 2024

fix infinite loop update checker in a specific condition

b4f80e6

close issue httpie#1527

Ousret added a commit to Ousret/httpie that referenced this issue Apr 5, 2024

fix infinite loop update checker in a specific condition

eb93eab

close issue httpie#1527

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Request is fired to packages.httpie.io on every invocation of `http` #1527

Request is fired to packages.httpie.io on every invocation of `http` #1527

thenbe commented Sep 4, 2023 •

edited

jkbrzt commented Sep 8, 2023

tgrushka commented Feb 25, 2024

OmichronAgain commented Mar 30, 2024

Ousret commented Apr 4, 2024

thenbe commented Apr 4, 2024

OmichronAgain commented Apr 4, 2024

Ousret commented Apr 4, 2024

Ousret commented Apr 4, 2024

OmichronAgain commented Apr 4, 2024

Ousret commented Apr 5, 2024

OmichronAgain commented Apr 5, 2024

Request is fired to packages.httpie.io on every invocation of http #1527

Request is fired to packages.httpie.io on every invocation of http #1527

Comments

thenbe commented Sep 4, 2023 • edited

Checklist

Minimal reproduction code and steps

Current result

Expected result

Debug output

Additional information, screenshots, or code examples

jkbrzt commented Sep 8, 2023

tgrushka commented Feb 25, 2024

OmichronAgain commented Mar 30, 2024

Ousret commented Apr 4, 2024

thenbe commented Apr 4, 2024

OmichronAgain commented Apr 4, 2024

Ousret commented Apr 4, 2024

Ousret commented Apr 4, 2024

OmichronAgain commented Apr 4, 2024

Ousret commented Apr 5, 2024

OmichronAgain commented Apr 5, 2024

Request is fired to packages.httpie.io on every invocation of `http` #1527

Request is fired to packages.httpie.io on every invocation of `http` #1527

thenbe commented Sep 4, 2023 •

edited