Updating rackerlabs master with upstream master changes #1
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
btorch
changed the title
While processing a WSGI request, if the remote client forcefully resets the connection while the request data is being read from the socket (e.g. HAProxy health check), the ECONNRESET socket error is not properly handled and may end up being logged in strerr or in a log file. #446
Directly depend on enum34, scoping requirement to < Python 3.4.
Hostname in /etc/hosts are not case-sensitive, this fixes HostsResolver() accordingly. #458 Co-Authored-By: Thomas Bechtold <tbechtold@suse.com>
…ads; Thanks to Jake Tesler #172
`greendns.HostsResolver.LINES_RE` doesn't admit the possibility of a comment starting at the beginning of a line. This has been ignored since `is_ipv4_addr()` and `is_ipv6_addr()` catch `dns.exception.SyntaxError` and return `False`. But in a runtime environment that encounters #413, `dns.exception.SyntaxError` is not caught, and the import fails. Changing '+' to '*' allows `HostsResolver._readlines()` to recognize and skip comment lines. greendns.HostsResolver._readlines(), a purely internal method, now returns an itertools generator rather than a list. Change relevant asserts to build a list before comparing.
If you have a pool with no free items, one greenthread blocked in pool.get(), and then you call pool.put(item), sometimes the put will block. This happens when the greenthread blocked in pool.get() has a pending timeout. The timeout's timer has fired, the call to throw() has been scheduled, but throw() has not actually run yet. In pool.put(), we see a waiting getter, so we do a blocking self.channel.put()... but when the getter runs, it unwinds its stack and does not take the item, leaving the caller of pool.put() blocked despite there being enough free space. This commit fixes that by (a) making LightQueue.put() and .get() work with 0-length queues, even with timeouts, and (b) checking for queue.Full in Pool.put() and handling it correctly. #495
The support of this version of python has been reintroduced [1]. Updating classifiers accordingly. 6441021
There are some cases where monkey_patching cannot always be performed early enough; such as sphinx autodoc importing. To help handle these cases, exceptions during patching are logged and execution is allowed to continue. This fixed an issue in OpenStack Manila docs generation caused by an upgrade to eventlet 0.35.0.
Close #821 For further details please see: - https://lore.kernel.org/all/20231103092706.6rw2ehuigxfdvhlc@lion.mk-sys.cz/ - https://bugzilla.suse.com/show_bug.cgi?id=1216858 - #821
If you're using eventlet as a web server, it's not unlikely that you'll be using eventlet.Timeouts at some point in your application callable or the response iterator that's returned. If they escape, don't let that blow up the whole worker greenthread, but treat it like other exceptions.
Django uses this for example.
Fix for (CVE-2023-29483) and handling of truncated exceptions in greendns.py provided by Bob Halley from https://github.com/rthalley/eventlet/tree/tudoor Do not eat legitimate Truncated exceptions. --------- Co-authored-by: Bob Halley <halley@play-bow.org> Co-authored-by: Hervé Beraud <hberaud@redhat.com>
In distros, when building the package and attempting to install it, there may be a "debian" folder. This creates this issue: error: Multiple top-level packages discovered in a flat-layout: ['debian', 'eventlet']. which is fixed by this patch. Please merge it to make it easier for package maintainers.
Co-authored-by: Itamar Turner-Trauring <itamar@pythonspeed.com>
This patch aims to add some convenience helpers
to help during debug session related to asyncio
and threading.
Those can be used in any running context.
As example, those helper could be used during pdb sessions
like this:
```
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
PDB post_mortem (IO-capturing turned off)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> /home/dev/app/tests/asyncio_test.py(235)go()
-> async def go():
(Pdb) import eventlet
(Pdb) pp eventlet.debug.format_threads_info()
('THREADS:\n'
'{139916325074752: <_MainThread(MainThread, started
139916325074752)>}')
(Pdb) pp eventlet.debug.format_asyncio_info()
('TASKS:\n'
"{<Task pending name='Task-1' coro=<Hub.run.<locals>.async_run()
running at "
'/home/dev/app/eventlet/hubs/asyncio.py:141>
cb=[_run_until_complete_cb() at '
'/usr/local/lib/python3.12/asyncio/base_events.py:181] created at '
'/usr/local/lib/python3.12/asyncio/tasks.py:695>}\n'
'EVENTLOOP: <_UnixSelectorEventLoop running=True closed=False
debug=True>')
```
The previous examples pretty print both helpers outputs.
This sentence of the file README differ from the eventlet doc https://eventlet.readthedocs.io/en/latest/
* Fix wsgi.server shutdown for in-flight requests While doing some work on OpenStack Swift, I noticed that in-flight requests were getting their sockets closed by eventlet when the listen socket was closed. In that case, there is a server process which bound the listen socket and N fork'ed off worker processes which all run the wsgi.server loop calling accept() and handling requests. I noticed that nothing was ever setting connection state to STATE_REQUEST (i.e. unused constant in previous patch). Upon closer inspection, it turns out that the socket cleanup code for STATE_IDLE sockets got applied to the socket with an in-progress request. This caused code using the socket to get an EPIPE and exit the request's greenthread without completing the request. With some instrumentation in the code, without this patch's fix, the new test observed the code doing this: (7544) wsgi starting up on http://127.0.0.1:36745 (7544) calling accept (7544) accept got <eventlet.greenio.base.GreenSocket object at 0x7f7650d73290>, ('127.0.0.1', 50194) (7544) accepted ('127.0.0.1', 50194) (7544) completed req ('127.0.0.1', 50194) (7544) calling accept (7544) got exception 22 (7544) re-raising! (7544) wsgi exiting, calling pool.waitall() Traceback (most recent call last): File "/tmp/swiftstack-eventlet/eventlet/wsgi.py", line 602, in handle_one_response write(b'') File "/tmp/swiftstack-eventlet/eventlet/wsgi.py", line 541, in write wfile.flush() File "/usr/lib64/python2.7/socket.py", line 303, in flush self._sock.sendall(view[write_offset:write_offset+buffer_size]) File "/tmp/swiftstack-eventlet/eventlet/greenio/base.py", line 403, in sendall tail = self.send(data, flags) File "/tmp/swiftstack-eventlet/eventlet/greenio/base.py", line 397, in send return self._send_loop(self.fd.send, data, flags) File "/tmp/swiftstack-eventlet/eventlet/greenio/base.py", line 384, in _send_loop return send_method(data, *args) error: [Errno 32] Broken pipe 127.0.0.1 - - [31/Oct/2019 01:08:05] "PUT /foo-bar HTTP/1.1" 200 0 0.000696 (7544) wsgi exited, is_accepting=True In the new tests, not having the fix manifests as all greenthreads exiting before the client finishes the in-flight request. For the WebsocketWSGI app, it has a private class attribute that tells the wsgi.server() code to consider its "reqeusts" always idle, which I believe was the primary intent of the original patch. * Fix tests for python 3.4 ...which apparently won't let you interpolate ints into bytestrings. Also remove some extraneous comments in the test. * Review comment fixes. Reduce chance for regression in future and show that client socket is unusable after an in-flight requet is finished. * Make Websocket apps work when wrapped * EBADF means accept() will never work ever again the wsgi module stop to consider bad file descriptors (EBADF) (closed sockets) as acceptable/recoverable errors. This kind of error is not recoverable from a server connection perspective. --------- Co-authored-by: Darrell Bishop <darrell@swiftstack.com> Co-authored-by: Itamar Turner-Trauring <itamar@pythonspeed.com> Co-authored-by: Hervé Beraud <hberaud@redhat.com>
#934) * Start of macOS runners. * macOS can't do Docker. * Can't use YAML anchors * Disable kqueue for now. * Unbreak kqueue in asyncio hub. * fork() is only usable (barely) on Linux * Mark test as known failure on macOS. * Less brittle test that also passes on macOS. * Mark more tests as known failures on macOS. * Fix issue on asyncio hub on macOS. * Don't leak sockets in xfail tests on macOS. * Fix too-long line. * Keep old test run names --------- Co-authored-by: Itamar Turner-Trauring <itamar@pythonspeed.com>
Assert statements are completely eliminated when the python interpreter is ran with the optimization flags [1]. Those checks are not an option and should be executed not matter the execution context. This patch refactor those checks to not rely anymore on the assert statement. This patch is mainly focused on hub checks. Other assert statements are still present, I'll refactor them later. Refactoring this kind of code ensure to not waste time managing issues and debugging bugs related to the optimization features of the interpreter. [1] https://docs.python.org/3/using/cmdline.html#cmdoption-O
…yncio (#930) * Fix and test for asyncio's internal thread pool. * Document the change. * Fix name and skip. * Make env_tpool_negative pass with asyncio again. * Use the correct API name. --------- Co-authored-by: Itamar Turner-Trauring <itamar@pythonspeed.com> Co-authored-by: Hervé Beraud <hberaud@redhat.com>
* Test for asyncio DNS lookups. * Make sure asyncio hub DNS APIs don't use greendns. * Lint fix --------- Co-authored-by: Itamar Turner-Trauring <itamar@pythonspeed.com>
Co-authored-by: Itamar Turner-Trauring <itamar@pythonspeed.com> Co-authored-by: Hervé Beraud <hberaud@redhat.com>
* Update changelog for version 0.36.0 Related to #940 --------- Co-authored-by: Itamar Turner-Trauring <itamar@itamarst.org>
The pyevent hub was removed one year ago, however the doc still contains a reference to it. This reference is empty and can let think that this hub is still present. Lets remove this reference.
Improve the general index by adding a contribution guide, changelog, etc. Refactor sections to host various topics and to generate a more sexy table of context which will be more user friendly. Refactor formatting heading levels. Make Asyncio warning message reusable and centralized. Give more visibility to the Asyncio topic.
* drop header keys with underscores * use a dedicated formalize_key_naming function for header normalisation * adjust tests to comply with the new header security checks
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.