-
-
Notifications
You must be signed in to change notification settings - Fork 86
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
FastAPI JSON weird #95
Comments
I need more info on how you create the Things, since I cannot recreate: import uuid
from typing import List
import databases
import pydantic
import pytest
import sqlalchemy
from fastapi import FastAPI
from starlette.testclient import TestClient
import ormar
from tests.settings import DATABASE_URL
app = FastAPI()
database = databases.Database(DATABASE_URL, force_rollback=True)
metadata = sqlalchemy.MetaData()
app.state.database = database
@app.on_event("startup")
async def startup() -> None:
database_ = app.state.database
if not database_.is_connected:
await database_.connect()
@app.on_event("shutdown")
async def shutdown() -> None:
database_ = app.state.database
if database_.is_connected:
await database_.disconnect()
class BaseMeta(ormar.ModelMeta):
metadata = metadata
database = database
class Thing(ormar.Model):
class Meta(BaseMeta):
tablename = "things"
id: uuid.UUID = ormar.UUID(primary_key=True, default=uuid.uuid4)
name: str = ormar.Text(default="")
js: pydantic.Json = ormar.JSON()
@app.get("/things", response_model=List[Thing])
async def read_things():
return await Thing.objects.order_by("name").all()
@app.get("/things_with_sample", response_model=List[Thing])
async def read_things():
await Thing(name="b", js=["asdf", "asdf", "bobby", "nigel"]).save()
await Thing(name="a", js="[\"lemon\", \"raspberry\", \"lime\", \"pumice\"]").save()
return await Thing.objects.order_by("name").all()
@app.post("/things", response_model=Thing)
async def create_things(thing: Thing):
thing = await thing.save()
return thing
@pytest.fixture(autouse=True, scope="module")
def create_test_database():
engine = sqlalchemy.create_engine(DATABASE_URL)
metadata.create_all(engine)
yield
metadata.drop_all(engine)
def test_read_main():
client = TestClient(app)
with client as client:
response = client.get("/things_with_sample")
assert response.status_code == 200
# check if raw response not double encoded
assert '["lemon","raspberry","lime","pumice"]' in response.text
# parse json and check that we get lists not strings
resp = response.json()
assert resp[0].get("js") == ["lemon", "raspberry", "lime", "pumice"]
assert resp[1].get("js") == ["asdf", "asdf", "bobby", "nigel"]
# create a new one
response = client.post("/things", json={"js": ["test", "test2"], "name": "c"})
assert response.json().get("js") == ["test", "test2"]
# get all with new one
response = client.get("/things")
assert response.status_code == 200
assert '["test","test2"]' in response.text
resp = response.json()
assert resp[0].get("js") == ["lemon", "raspberry", "lime", "pumice"]
assert resp[1].get("js") == ["asdf", "asdf", "bobby", "nigel"]
assert resp[2].get("js") == ["test", "test2"] |
Huhhh. It looks like it depends on how you set the JSON field. Consider:
and a subsequent GET from /things yields
The after-constructor string behavior also shows up when I afterwards do e.g.
Glad to know it's intended to work how I expected it to work, btw. |
Still cannot reproduce :) import uuid
from typing import List
import databases
import pydantic
import pytest
import sqlalchemy
from fastapi import FastAPI
from starlette.testclient import TestClient
import ormar
from tests.settings import DATABASE_URL
app = FastAPI()
database = databases.Database(DATABASE_URL, force_rollback=True)
metadata = sqlalchemy.MetaData()
app.state.database = database
@app.on_event("startup")
async def startup() -> None:
database_ = app.state.database
if not database_.is_connected:
await database_.connect()
@app.on_event("shutdown")
async def shutdown() -> None:
database_ = app.state.database
if database_.is_connected:
await database_.disconnect()
class BaseMeta(ormar.ModelMeta):
metadata = metadata
database = database
class Thing(ormar.Model):
class Meta(BaseMeta):
tablename = "things"
id: uuid.UUID = ormar.UUID(primary_key=True, default=uuid.uuid4)
name: str = ormar.Text(default="")
js: pydantic.Json = ormar.JSON()
@app.get("/things", response_model=List[Thing])
async def read_things():
return await Thing.objects.order_by("name").all()
@app.get("/things_with_sample", response_model=List[Thing])
async def read_things_sample():
await Thing(name="b", js=["asdf", "asdf", "bobby", "nigel"]).save()
await Thing(name="a", js="[\"lemon\", \"raspberry\", \"lime\", \"pumice\"]").save()
return await Thing.objects.order_by("name").all()
@app.get("/things_with_sample_after_init", response_model=Thing)
async def read_things_init():
thing1 = Thing()
thing1.name = "d"
thing1.js = ["js", "set", "after", "constructor"]
await thing1.save()
return thing1
@app.put("/update_thing", response_model=Thing)
async def update_things(thing: Thing):
thing.js = ["js", "set", "after", "update"] # type: ignore
await thing.update()
return thing
@app.post("/things", response_model=Thing)
async def create_things(thing: Thing):
thing = await thing.save()
return thing
@pytest.fixture(autouse=True, scope="module")
def create_test_database():
engine = sqlalchemy.create_engine(DATABASE_URL)
metadata.create_all(engine)
yield
metadata.drop_all(engine)
def test_read_main():
client = TestClient(app)
with client as client:
response = client.get("/things_with_sample")
assert response.status_code == 200
# check if raw response not double encoded
assert '["lemon","raspberry","lime","pumice"]' in response.text
# parse json and check that we get lists not strings
resp = response.json()
assert resp[0].get("js") == ["lemon", "raspberry", "lime", "pumice"]
assert resp[1].get("js") == ["asdf", "asdf", "bobby", "nigel"]
# create a new one
response = client.post("/things", json={"js": ["test", "test2"], "name": "c"})
assert response.json().get("js") == ["test", "test2"]
# get all with new one
response = client.get("/things")
assert response.status_code == 200
assert '["test","test2"]' in response.text
resp = response.json()
assert resp[0].get("js") == ["lemon", "raspberry", "lime", "pumice"]
assert resp[1].get("js") == ["asdf", "asdf", "bobby", "nigel"]
assert resp[2].get("js") == ["test", "test2"]
response = client.get("/things_with_sample_after_init")
assert response.status_code == 200
resp = response.json()
assert resp.get("js") == ["js", "set", "after", "constructor"]
# test new with after constructor
response = client.get("/things")
resp = response.json()
assert resp[0].get("js") == ["lemon", "raspberry", "lime", "pumice"]
assert resp[1].get("js") == ["asdf", "asdf", "bobby", "nigel"]
assert resp[2].get("js") == ["test", "test2"]
assert resp[3].get("js") == ["js", "set", "after", "constructor"]
response = client.put("/update_thing", json=resp[3])
assert response.status_code == 200
resp = response.json()
assert resp.get("js") == ["js", "set", "after", "update"]
# test new with after constructor
response = client.get("/things")
resp = response.json()
assert resp[0].get("js") == ["lemon", "raspberry", "lime", "pumice"]
assert resp[1].get("js") == ["asdf", "asdf", "bobby", "nigel"]
assert resp[2].get("js") == ["test", "test2"]
assert resp[3].get("js") == ["js", "set", "after", "update"] |
Looks like it's dependent on the
Now, I'd be kindof ok with that behavior - at least there's a fix. But it sounds like maybe it wasn't intentional. Also, the following yields different outputs, when that seems unlikely to be the expected result:
(I've removed some I/O errors I keep getting in the console, which don't seem to correlate with what I type and don't seem to affect anything.) |
Further related strange behavior:
Specifically, it looks like I note that the database appears to store the two cases as "a string containing json" and "a string containing a string containing json". |
Re: db, like, if I do
|
Also, in testing just now I've discovered that you can set an invalid value on a |
Clarification: you can set an invalid value after creation. |
Ok i will check that, thanks for reporting. As for choices is it on json field or other one? If it's not the json column can you please open another ticket with a sample to reproduce? Thanks for catching that! 🙂 |
Ok, both should be fixed in 0.9.3. |
Both seem fixed, thanks! Question, though - before, I could leave |
Yeah, it was changed in one of the old version to be in line with As for the rules below is the section from the docs overview: All fields are required unless one of the following is set:
Also relation fields by default are nullable. So in your case you should just set |
Great, that works. Thanks! I'll open more tickets if I find anything else
weird or wrong.
…On Mon, Feb 15, 2021, 2:10 PM collerek ***@***.***> wrote:
Yeah, it was changed in one of the old version to be in line with pydantic
where you have to specifically set field to optional or provide a default
value for the field to be not required. See that i forgot to update the
nullable part of docs, thanks for the tip, will change that.
As for the rules below is the section from the docs overview:
All fields are required unless one of the following is set:
- nullable - Creates a nullable column. Sets the default to None.
- default - Set a default value for the field.
- server_default - Set a default value for the field on server side
(like sqlalchemy's func.now()).
- primary key with autoincrement - When a column is set to primary key
and autoincrement is set on this column. Autoincrement is set by default on
int primary keys.
- pydantic_only - Field is available only as normal pydantic field,
not stored in the database.
So in your case you should just set nullable=True on js Field.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
<#95 (comment)>, or
unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAXCNTPSMFO2HFNLPX4OI4DS7FWTHANCNFSM4XMBS25Q>
.
|
So, I'm not sure which library bears responsibility here, but I'll start with ormar and you can forward me somewhere else if the problem lies elsewhere. Consider the following code:
What I get when I call this endpoint is e.g.
Note how rather than being JSON, the
js
field is a plain string containing JSON. Is this on purpose? Does it HAVE to be that way? It seems to me like it would make more sense for a JSON field, when its container is serialized to JSON, to just...be JSON. (I note thatthing.json()
preserves the "convert json to string" behavior.) Is there an easy way around this behavior, perhaps a flag or setting? Is this actually a result of a different library?The text was updated successfully, but these errors were encountered: