Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feature: support JSON(typed_dict=TypedDict() ) Input, Output #3748

Closed
KimSoungRyoul opened this issue Apr 9, 2023 · 2 comments
Closed

feature: support JSON(typed_dict=TypedDict() ) Input, Output #3748

KimSoungRyoul opened this issue Apr 9, 2023 · 2 comments

Comments

@KimSoungRyoul
Copy link
Contributor

KimSoungRyoul commented Apr 9, 2023

Feature request

to be

class IAmSchema(BaseModel):
    a: list[float]
    b: list[float]
    c: list[float]
    d: Optional[float] # float | None



@svc.api(
    input=JSON( pydantic_model=IAmSchema ),
    output=JSON.from_sample({"qqqq":"wwww"}), 
)
async def predict_json(i_am_schema: ASchema) -> Dict[str,Any]:
    result = await runner.async_run(torch.tensor(list(i_am_schema.dict().values())))

    return {"qqqq": result}

as is

class IAmSchema(t.TypedDict, total=False):
    a: list[float]
    b: list[float]
    c: list[float]
    d: NotRequried[float] # use NotRequired instead of Optional
    # Optional is rejected Spec in TypedDict (pep-0655)
    #  https://peps.python.org/pep-0655/#special-syntax-around-the-key-of-a-typeddict-item


@svc.api(
    input=JSON( 
        typeddict=IAmSchema, # <--- use TypedDict
    ),
    output=JSON.from_sample({"qqqq":"wwww"})), # <-- JSON IODescriptor cann't specify specifications in OAS3.
)
async def predict_json(i_am_schema: IAmSchema) -> Dict[str, Ant]:
     result = await runner.async_run(torch.tensor(list(i_am_schema.values()))

    return {"qqqq2222": result}

Motivation

If we use Pydantic just to specify a type schema without validation, the logic of xx.dict() -> Pydantic.parse_obj() unnecessary layer which can affect performance. In this case, using TypedDict seems necessary.

here is POC #3749
#3809

I'd like to hear your thoughts on TypedDict

Other

No response

@aarnphm
Copy link
Contributor

aarnphm commented Apr 9, 2023

If we want to support TypedDict then it should just be supported by JSON and using the typing system, to some extend (but parsing types are costly, esp for IO descriptor as it will affect users throughput).

TypedDict is only available from 3.8+ so this means we will have to depends on typing_extensions.

From a quick glance from the implementation, it seems like you are just using Pydantic, so I don't see the point of having this as a part of the core library.

The core usage of Pydantic is for validation. So if we want to support TypedDict and attrs class to an extend, it should probably be under the JSON IO descriptor.

@KimSoungRyoul KimSoungRyoul changed the title feature: support TypedDict Input, Output feature: support JSON(typed_dict=TypedDict() ) Input, Output Apr 13, 2023
KimSoungRyoul added a commit to KimSoungRyoul/BentoML that referenced this issue Apr 30, 2023
KimSoungRyoul added a commit to KimSoungRyoul/BentoML that referenced this issue Apr 30, 2023
KimSoungRyoul added a commit to KimSoungRyoul/BentoML that referenced this issue Apr 30, 2023
KimSoungRyoul added a commit to KimSoungRyoul/BentoML that referenced this issue May 1, 2023
* (pyproject.toml) add dependency "io-json = [... "type-extensions"]"
* (json.py) separate code block service.py(pydantic) and service.py(TypedDict)
* (json.py) add validate "typeddict and pydantic_model are mutually exclusive"
* (json.py) "if sys.version_info >= (3, 8):"
* (json.py) explicit check () "if typeddict is not None and not is_typeddict(typeddict)"
KimSoungRyoul added a commit to KimSoungRyoul/BentoML that referenced this issue May 1, 2023
* (json.py) check legacy_typeddict
  BadInput("You should use `typing_extensions.TypedDict` instead of `typing.TypedDict` with Python < 3.11.")

* (json.py) refactor openapi_schema() & openapi_components()
* (tests/unit/_internal/io/test_json.py) typeddict testcase
* (utils.py) refactor typed_dict_to_schema() -> typed_dict_to_dict()
KimSoungRyoul added a commit to KimSoungRyoul/BentoML that referenced this issue May 1, 2023
* (json.py) remove pydantic.is_legacy_typeddict()

* (utils.py) remove useless import

* (pyproject.toml) fix "type-extensions" to "typing-extensions"
KimSoungRyoul added a commit to KimSoungRyoul/BentoML that referenced this issue May 1, 2023
* (json.py) remove pydantic.is_legacy_typeddict()

* (utils.py) remove useless import

* (pyproject.toml) fix "type-extensions" to "typing-extensions"
KimSoungRyoul added a commit to KimSoungRyoul/BentoML that referenced this issue May 2, 2023
* (json.py) t.Type["TypedDict"] ->  type[t.TypedDict]
KimSoungRyoul added a commit to KimSoungRyoul/BentoML that referenced this issue May 2, 2023
* (json.py) update error msg & docs example
KimSoungRyoul added a commit to KimSoungRyoul/BentoML that referenced this issue May 2, 2023
KimSoungRyoul added a commit to KimSoungRyoul/BentoML that referenced this issue May 2, 2023
KimSoungRyoul added a commit to KimSoungRyoul/BentoML that referenced this issue May 4, 2023
KimSoungRyoul added a commit to KimSoungRyoul/BentoML that referenced this issue May 14, 2023
KimSoungRyoul added a commit to KimSoungRyoul/BentoML that referenced this issue May 29, 2023
@KimSoungRyoul
Copy link
Contributor Author

This issue seems to be pointless in the current 1.2 version.
and
if need necessary, a new discussion issue should be created in version 1.2.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants