Skip to content

Commit

Permalink
Add requests API compatibility in README
Browse files Browse the repository at this point in the history
Bump version
  • Loading branch information
ml31415 committed Apr 18, 2024
1 parent c97ad81 commit 0e55d8a
Show file tree
Hide file tree
Showing 3 changed files with 43 additions and 42 deletions.
81 changes: 41 additions & 40 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,68 +5,72 @@

# geventhttpclient

A high performance, concurrent HTTP client library for python using
A [high performance](https://github.com/geventhttpclient/geventhttpclient/blob/master/README.md#Benchmarks),
concurrent HTTP client library for python using
[gevent](http://gevent.org).

`gevent.httplib` support for patching `http.client` was removed in
[gevent 1.0](https://github.com/surfly/gevent/commit/b45b83b1bc4de14e3c4859362825044b8e3df7d6),
**geventhttpclient** now provides that missing functionality.
`geventhttpclient` now provides that missing functionality.

**geventhttpclient** uses a fast [http parser](https://github.com/nodejs/llhttp),
`geventhttpclient` uses a fast [http parser](https://github.com/nodejs/llhttp),
written in C.

**geventhttpclient** has been specifically designed for high concurrency,
`geventhttpclient` has been specifically designed for high concurrency,
streaming and support HTTP 1.1 persistent connections. More generally it is
designed for efficiently pulling from REST APIs and streaming APIs
like Twitter's.

Safe SSL support is provided by default. **geventhttpclient** depends on
Safe SSL support is provided by default. `geventhttpclient` depends on
the certifi CA Bundle. This is the same CA Bundle which ships with the
Requests codebase, and is derived from Mozilla Firefox's canonical set.

As of version 2.1, Python versions 3.9+ are (fully) supported.

A simple example:
Since version 2.3, `geventhttpclient` features a largely `requests`
compatible interface. It covers basic HTTP usage including cookie
management, form data encoding or decoding of compressed data,
but otherwise isn't as feature rich as the original `requests`. For
simple use-cases, it can serve as a drop-in replacement.

```python
#!/usr/bin/python
import geventhttpclient as requests
requests.get("https://github.com").text
requests.post("http://httpbingo.org/post", data="asdfasd").json()

from geventhttpclient import Session
s = Session()
s.get("http://httpbingo.org/headers").json()
s.get("https://github.com").content
```

This interface builds on top of the lower level `HTTPClient`.

```python
from geventhttpclient import HTTPClient
from geventhttpclient.url import URL

url = URL("http://gevent.org/")

client = HTTPClient(url.host)

# issue a get request
response = client.get(url.request_uri)

# read status_code
response.status_code

# read response body
body = response.read()

# close connections
client.close()
```

## httplib compatibility and monkey patch

**geventhttpclient.httplib** module contains classes for drop in
replacement of http.client connection and response objects.
If you use http.client directly you can replace the **httplib** imports
by **geventhttpclient.httplib**.
`geventhttpclient.httplib` module contains classes for drop in
replacement of `http.client` connection and response objects.
If you use http.client directly you can replace the `httplib` imports
by `geventhttpclient.httplib`.

```python
# from http.client import HTTPConnection
from geventhttpclient.httplib import HTTPConnection
```

If you use **httplib2**, **urllib** or **urllib2**; you can patch **httplib** to
use the wrappers from **geventhttpclient**.
For **httplib2**, make sure you patch before you import or the `super`
calls will fail.
If you use `httplib2`, `urllib` or `urllib2`; you can patch `httplib` to
use the wrappers from `geventhttpclient`. For `httplib2`, make sure you
patch before you import or the `super()` calls will fail.

```python
import geventhttpclient.httplib
Expand All @@ -77,13 +81,11 @@ import httplib2

## High Concurrency

`HTTPClient` has connection pool built in and is greenlet safe by design.
`HTTPClient` has a connection pool built in and is greenlet safe by design.
You can use the same instance among several greenlets. It is the low
level building block of this library.

```python
#!/usr/bin/env python

import gevent.pool
import json

Expand Down Expand Up @@ -132,16 +134,14 @@ client.close()

## Streaming

**geventhttpclient** supports streaming.
Response objects have a read(N) and readline() method that read the stream
incrementally.
See *examples/twitter_streaming.py* for pulling twitter stream API.
`geventhttpclient` supports streaming. Response objects have a `read(n)` and
`readline()` method that read the stream incrementally.
See [examples/twitter_streaming.py](https://github.com/geventhttpclient/geventhttpclient/blob/master/examples/twitter_streaming.py)
for pulling twitter stream API.

Here is an example on how to download a big file chunk by chunk to save memory:

```python
#!/usr/bin/env python

from geventhttpclient import HTTPClient, URL

url = URL("http://127.0.0.1:80/100.dat")
Expand All @@ -159,11 +159,12 @@ with open("/tmp/100.dat", "w") as f:

## Benchmarks

The benchmark does 10000 get requests against a local nginx server in the default
The benchmark runs 10000 `GET` requests against a local nginx server in the default
configuration with a concurrency of 10. See `benchmarks` folder. The requests per
second for a couple of popular clients is given in the table below. Please read
[benchmarks/README.md](https://github.com/geventhttpclient/geventhttpclient/blob/master/benchmarks/README.md)
for more details. Also note, HTTPX is better be used with asyncio, not gevent.
for more details. Also note, [HTTPX](https://www.python-httpx.org/) is better be
used with `asyncio`, not `gevent`.

| HTTP Client | RPS |
|--------------------|--------|
Expand All @@ -178,7 +179,7 @@ for more details. Also note, HTTPX is better be used with asyncio, not gevent.
## License

This package is distributed under the [MIT license](https://github.com/geventhttpclient/geventhttpclient/blob/master/LICENSE-MIT).
Previous versions of geventhttpclient used http_parser.c, which in turn
was based on `src/http/ngx_http_parse.c` from NGINX, copyright Igor Sysoev,
Joyent, Inc., and other Node contributors. For more information, see
Previous versions of geventhttpclient used `http_parser.c`, which in turn was
based on `http/ngx_http_parse.c` from [NGINX](https://nginx.org), copyright Igor
Sysoev, Joyent, Inc., and other Node contributors. For more information, see
http://github.com/joyent/http-parser
2 changes: 1 addition & 1 deletion geventhttpclient/__init__.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# package

__version__ = "2.2.1" # dont forget to update version in pyproject.toml as well
__version__ = "2.3.0" # dont forget to update version in pyproject.toml as well

from geventhttpclient.api import delete, get, head, options, patch, post, put, request
from geventhttpclient.client import HTTPClient
Expand Down
2 changes: 1 addition & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ build-backend = "setuptools.build_meta"

[project]
name = "geventhttpclient"
version = "2.2.1" # dont forget to update version __init__.py as well
version = "2.3.0" # dont forget to update version __init__.py as well
description = "HTTP client library for gevent"
readme = "README.md"
requires-python = ">=3.9"
Expand Down

0 comments on commit 0e55d8a

Please sign in to comment.