Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ASGI Support #532

Merged
merged 101 commits into from Dec 4, 2019
Merged
Changes from 1 commit
Commits
Show all changes
101 commits
Select commit Hold shift + click to select a range
5eaf1a1
Trash
dmanchon Jun 13, 2019
853f476
requirements
dmanchon Jun 13, 2019
d023abb
Tests + changes to factory
masipcat Jun 14, 2019
3bd33e9
hyper
dmanchon Jun 14, 2019
ea8c6dd
Clean a bit
dmanchon Jun 14, 2019
4acf812
make hypercorn work
masipcat Jun 14, 2019
f59e459
Using custom Request instead of aiohttp.Request
masipcat Jun 14, 2019
b11dd0d
POST fixed. Molotov test
dmanchon Jun 14, 2019
032f68c
Fix testclient fixture
masipcat Jun 14, 2019
ab8a126
Small changes
masipcat Jun 15, 2019
898142b
websockets with asgi draft
dmanchon Jun 15, 2019
f010e3a
websocket close handling
dmanchon Jun 15, 2019
e2c38c6
Move requests implementations to a module
dmanchon Jun 15, 2019
1827366
fix test
vangheem Jun 15, 2019
55a66d8
Fixed GuillotinaRequest.query_string
masipcat Jun 15, 2019
62fff01
Checkpoint. Almost all tests are green
masipcat Jun 15, 2019
093e94f
Fix ws tests
dmanchon Jun 15, 2019
d8d6a13
Implement IRequest record method on asgi
dmanchon Jun 15, 2019
27e4b2a
Deleted aiohttp request
masipcat Jun 15, 2019
db9b798
Fix debug headers handler
dmanchon Jun 16, 2019
5ae7797
Merge branch 'master' into asgi-support
masipcat Jun 16, 2019
89fafb3
Merge branch 'asgi-support' of github.com:plone/guillotina into asgi-…
masipcat Jun 16, 2019
05d10ef
Fix deleted ws import
masipcat Jun 16, 2019
309dce8
Fixed AsgiStreamReader bug
masipcat Jun 16, 2019
2fc67b2
More fixes
masipcat Jun 16, 2019
9c944af
Small fix
dmanchon Jun 16, 2019
df15f81
Simulate incoming request in chunks for uploads
dmanchon Jun 16, 2019
826dd32
Remove starlette dep
dmanchon Jun 16, 2019
29dd329
flake8
dmanchon Jun 16, 2019
d72d373
Almost there
masipcat Jun 16, 2019
1c4d5c7
All tests green
masipcat Jun 16, 2019
69adf04
Now should be all green
masipcat Jun 16, 2019
7952fb0
Trying to fix failing tests in travis
masipcat Jun 16, 2019
7c10a56
I hope now is fixed...
masipcat Jun 16, 2019
76963a7
Small fix
masipcat Jun 16, 2019
5269254
Revert "Trying to fix failing tests in travis"
masipcat Jun 16, 2019
49ab931
Fix tests when runnign with DB_SCHEMA != public
masipcat Jun 16, 2019
c62bd3a
update to last asgi test client
dmanchon Jun 17, 2019
2a0146e
Updated async-asgi-testclient
masipcat Jun 17, 2019
223f083
Merge branch 'master' into asgi-support
masipcat Jun 17, 2019
9eec6ec
Small changes
masipcat Jun 17, 2019
eee7568
Fix merge
masipcat Jun 17, 2019
70a2a0f
Merge branch 'master' into asgi-support
masipcat Jun 18, 2019
ed3e7eb
Configured pg v10 in pytest-docker-fixtures + small change in fixtures
masipcat Jun 18, 2019
99e012b
Merge branch 'master' into asgi-support
masipcat Jun 18, 2019
f19412a
Merge branch 'master' into asgi-support
masipcat Jun 19, 2019
1c6dca7
Clean up unused methods
masipcat Jun 19, 2019
841333d
Merge branch 'master' into asgi-support
masipcat Jun 19, 2019
8ecb6b8
Rearrenged code and reduced code that depends on aiohttp
masipcat Jun 19, 2019
3c7eb26
Merge branch 'master' into asgi-support
masipcat Jun 19, 2019
efe666b
Updated changelog
masipcat Jun 19, 2019
4f0714e
Fix pg catalog tests
masipcat Jun 21, 2019
c8fe97c
Remove aiohttp dependecy for websockets
dmanchon Jun 22, 2019
dcba06c
Remove aiohttp dependecy for websockets
dmanchon Jun 22, 2019
7aafe6f
Remove aiohttp dependecy for request
dmanchon Jun 22, 2019
30b58fa
Flake8
dmanchon Jun 22, 2019
8fe4e10
Replaced 'loop' fixture from 'aiohttp' for 'event_loop' from 'pytest-…
masipcat Jun 22, 2019
ca8be11
Merge branch 'master' into asgi-support
masipcat Jun 22, 2019
2086715
Fix cockroach fixture
masipcat Jun 22, 2019
f6b49ba
Reduced aiohttp dependence. TODO: traversal/router and CORS
masipcat Aug 29, 2019
147c233
BOOM! Merge branch 'master' into asgi-support
masipcat Aug 29, 2019
fbb0b76
Lot of fixes
masipcat Aug 29, 2019
b375401
Fixed flake8 and mypy
masipcat Aug 29, 2019
133a30e
black
masipcat Aug 29, 2019
f7f6669
Added some tests and cleaned unused code
masipcat Aug 30, 2019
22c6a8a
Mypy
masipcat Aug 30, 2019
00eaacd
Asgi support: no aiohttp (#654)
vangheem Aug 31, 2019
699176e
Merge branch 'master' into asgi-support
masipcat Aug 31, 2019
168c560
Merge branch 'master' into asgi-support
masipcat Sep 3, 2019
80f25f1
isort
masipcat Sep 3, 2019
f73bef1
Merge branch 'master' into asgi-support
masipcat Sep 3, 2019
98f4c11
Merge branch 'master' into asgi-support
masipcat Sep 20, 2019
fbc08df
Merge branch 'master' into asgi-support
masipcat Sep 20, 2019
f1211c8
Documented how to use differents ASGI servers + small changes
masipcat Sep 22, 2019
272f331
Small fixes
masipcat Sep 22, 2019
307a31d
Merge branch 'master' into asgi-support
masipcat Sep 24, 2019
095125f
mypy-flake8
masipcat Sep 24, 2019
6726ce1
Merge branch 'asgi-support' of github.com:plone/guillotina into asgi-…
masipcat Sep 24, 2019
3203b0d
Changes and fixes
masipcat Sep 24, 2019
e5d6a90
Removed yarl
masipcat Sep 24, 2019
0265be1
Support for middlewares
masipcat Oct 2, 2019
e0955ef
Merge branch 'master' into asgi-support
masipcat Oct 2, 2019
0333e34
Black
masipcat Oct 2, 2019
61d7d20
Merge branch 'master' into asgi-support
masipcat Oct 18, 2019
49230f1
Updated Cython for python3.8 (required by uvloop)
masipcat Oct 18, 2019
515327a
fix uvloop
masipcat Oct 18, 2019
ed069ad
requested changes
masipcat Oct 18, 2019
12deec6
Merge branch 'master' into asgi-support
masipcat Oct 23, 2019
dc73ece
Removed python 3.8 in travis
masipcat Oct 23, 2019
f925322
Merge branch 'master' into asgi-support
masipcat Oct 25, 2019
b57cd01
Changed implementation of reify
masipcat Oct 25, 2019
88751c5
Merge branch 'master' into asgi-support
masipcat Nov 7, 2019
7ae3546
Merge branch 'master' into asgi-support
masipcat Nov 21, 2019
c94cd49
Fix some tests are skiped and mypy errors
masipcat Nov 21, 2019
6295819
Install extra 'testdata' in travis
masipcat Nov 21, 2019
a82df48
Merge branch 'master' into asgi-support
masipcat Nov 24, 2019
be427dd
Fixed tests
masipcat Nov 24, 2019
5547069
Merge branch 'master' into asgi-support
masipcat Nov 26, 2019
a7bedb5
Merge branch 'master' into asgi-support
masipcat Dec 4, 2019
827e228
Small fixes
masipcat Dec 4, 2019
52f4200
fix
masipcat Dec 4, 2019
File filter...
Filter file types
Jump to…
Jump to file or symbol
Failed to load files and symbols.

Always

Just for now

Using custom Request instead of aiohttp.Request

  • Loading branch information
masipcat committed Jun 14, 2019
commit f59e4596ba056d0b1ab14d39c1bd9b618d8a43f4
@@ -1,17 +1,228 @@
from aiohttp.test_utils import make_mocked_request
from aiohttp.streams import EmptyStreamReader
from aiohttp.web import StreamResponse
from aiohttp.helpers import reify
import asyncio
import multidict
import guillotina
from guillotina.request import Request
import os
import json
import yaml
from yarl import URL

from typing import Any, Iterator, Tuple


def headers_to_list(headers):
return [[k.encode(), v.encode()] for k, v in headers.items()]




class GuillotinaRequest(Request):

def __init__(self, scheme, method, path, raw_headers, payload, client_max_size: int=1024**2):
self._scheme = scheme
self._method = method
self._rel_url = URL(path)
self._raw_headers = raw_headers
self._payload = payload

self._client_max_size = client_max_size

self._state = {}
self._cache = {}

@reify
def rel_url(self):
return self._rel_url

# MutableMapping API

def __getitem__(self, key: str) -> Any:
return self._state[key]

def __setitem__(self, key: str, value: Any) -> None:
self._state[key] = value

def __delitem__(self, key: str) -> None:
del self._state[key]

def __len__(self) -> int:
return len(self._state)

def __iter__(self) -> Iterator[str]:
return iter(self._state)

@reify
def scheme(self):
return self._scheme

@reify
def method(self) -> str:
"""Read only property for getting HTTP method.
The value is upper-cased str like 'GET', 'POST', 'PUT' etc.
"""
return self._method

@reify
def version(self) -> Tuple[int, int]:
"""Read only property for getting HTTP version of request.
Returns aiohttp.protocol.HttpVersion instance.
"""
return self._version

@reify
def host(self) -> str:
"""Hostname of the request.
Hostname is resolved in this order:
- overridden value by .clone(host=new_host) call.
- HOST HTTP header
- socket.getfqdn() value
"""
host = self.headers.get("host")
return host

@reify
def url(self):
url = URL.build(scheme=self.scheme, host=self.host)
return url.join(self._rel_url)

@reify
def path(self) -> str:
"""The URL including *PATH INFO* without the host or scheme.
E.g., ``/app/blog``
"""
return self._rel_url.path

@reify
def path_qs(self) -> str:
"""The URL including PATH_INFO and the query string.
E.g, /app/blog?id=10
"""
return str(self._rel_url)

@reify
def raw_path(self) -> str:
""" The URL including raw *PATH INFO* without the host or scheme.
Warning, the path is unquoted and may contains non valid URL characters
E.g., ``/my%2Fpath%7Cwith%21some%25strange%24characters``
"""
return self._message.path

@reify
def query(self) -> 'MultiDictProxy[str]':
"""A multidict with all the variables in the query string."""
return self._rel_url.query

@reify
def query_string(self) -> str:
"""The query string in the URL.
E.g., id=10
"""
return self._rel_url.query_string

@reify
def headers(self) -> 'CIMultiDictProxy[str]':
"""A case-insensitive multidict proxy with all headers."""
headers = multidict.CIMultiDict()
# TODO: extend
for key, value in self._raw_headers:
headers.add(key.decode(), value.decode())
self._headers = headers
return headers

@reify
def raw_headers(self):
"""A sequence of pairs for all headers."""
return self._raw_headers

@reify
def content(self):
"""Return raw payload stream."""
return self._payload

@property
def has_body(self) -> bool:
"""Return True if request's HTTP BODY can be read, False otherwise."""
warnings.warn(
"Deprecated, use .can_read_body #2005",
DeprecationWarning, stacklevel=2)
return not self._payload.at_eof()

@property
def can_read_body(self) -> bool:
"""Return True if request's HTTP BODY can be read, False otherwise."""
return not self._payload.at_eof()

@reify
def body_exists(self) -> bool:
"""Return True if request has HTTP BODY, False otherwise."""
return type(self._payload) is not EmptyStreamReader

async def release(self) -> None:
"""Release request.
Eat unread part of HTTP BODY if present.
"""
while not self._payload.at_eof():
await self._payload.readany()

async def read(self) -> bytes:
"""Read request body if present.
Returns bytes object with full request content.
"""
if self._read_bytes is None:
body = bytearray()
while True:
chunk = await self._payload.readany()
body.extend(chunk)
if self._client_max_size:
body_size = len(body)
if body_size >= self._client_max_size:
raise HTTPRequestEntityTooLarge(
max_size=self._client_max_size,
actual_size=body_size
)
if not chunk:
break
self._read_bytes = bytes(body)
return self._read_bytes

async def text(self) -> str:
"""Return BODY as text using encoding from .charset."""
bytes_body = await self.read()
encoding = self.charset or 'utf-8'
return bytes_body.decode(encoding)

async def json(self, *, loads=json.loads) -> Any:
"""Return BODY as JSON."""
body = await self.text()
return loads(body)

def __repr__(self) -> str:
ascii_encodable_path = self.path.encode('ascii', 'backslashreplace') \
.decode('ascii')
return "<{} {} {} >".format(self.__class__.__name__,
self._method, ascii_encodable_path)

def __eq__(self, other: object) -> bool:
return id(self) == id(other)

async def _prepare_hook(self, response: StreamResponse) -> None:
return



class AsgiStreamReader(EmptyStreamReader):
"""Dummy implementation of StreamReader"""

@@ -56,6 +267,7 @@ def __init__(self):
# The config file is defined in the env var `CONFIG`
loop = asyncio.get_event_loop()
from guillotina.factory import make_app
import guillotina

config = os.getenv("CONFIG", None)
if settings:
@@ -68,23 +280,19 @@ def __init__(self):
return await make_app(settings=settings, loop=loop, server_app=self)

async def handler(self, scope, receive, send):
# Copy headers
headers = multidict.CIMultiDict()
raw_headers = scope["headers"]
for key, value in raw_headers:
headers.add(key.decode(), value.decode())

method = scope["method"]
path = scope["path"]

# Aiohttp compatible StreamReader
payload = AsgiStreamReader(receive)

request = make_mocked_request(method, path, headers=headers, payload=payload)
request = GuillotinaRequest(
scope["scheme"],
scope["method"],
scope["path"],
scope["headers"],
payload
)

# This is to fake IRequest interface
request.record = lambda x: None
request.__class__ = guillotina.request.Request

route = await self.app.router.resolve(request)
resp = await route.handler(request)
@@ -101,5 +309,9 @@ def __init__(self):
await send({"type": "http.response.body", "body": body.encode()})



from guillotina.factory.app import make_asgi_app

# asgi app singleton
app = AsgiApp()
app = make_asgi_app()

@@ -277,10 +277,18 @@ def __init__(self, request):


@pytest.fixture(scope='function')
async def app_client(guillotina_main):
app = guillotina_main
async def app_client(loop):
globalregistry.reset()
from guillotina.factory.app import make_asgi_app
app = make_asgi_app()

yield app, TestClient(app)
async with TestClient(app) as client:
g_app = app.app
#g_app = await app.startup(settings=get_db_settings(), loop=loop)
g_app.config.execute_actions()
load_cached_schema()
await _clear_dbs(g_app.root)
yield app, client


@pytest.fixture(scope='function')
ProTip! Use n and p to navigate between commits in a pull request.
You can’t perform that action at this time.