Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ability to parse JSON with arbitrary keys #19

Closed
jaredculp opened this issue Jul 27, 2023 · 5 comments
Closed

Ability to parse JSON with arbitrary keys #19

jaredculp opened this issue Jul 27, 2023 · 5 comments
Assignees
Labels
enhancement Enhancement to existing feature
Milestone

Comments

@jaredculp
Copy link

jaredculp commented Jul 27, 2023

Is it possible to enforce the presence of certain keys but ignore all others? My use case is parsing JSON payloads from an event bus where I want to ignore fields that may be present but that I don't care to include in the TypedDict.

This functionality would be similar to json-schema's additionalProperties: true keyword.

For example:

from typing import TypedDict, NotRequired
from trycast import trycast

class MyType(TypedDict):
    a: str
    b: NotRequired[str | None]

# currently fails validation because "c" is in the dict
trycast(MyType, {"a": "foo", "b": "bar", "c": "baz"})
@davidfstr
Copy link
Owner

Is it possible to enforce the presence of certain keys but ignore all others?

Not at the moment. As you demonstrated trycast currently requires that there be no extra keys when checking assignability to a TypedDict type.

However I actually think trycast's behavior should be changed to allow extra keys by default. Consider the following code which typechecks successfully with mypy:

from typing import *

class Packet(TypedDict):
    type: str
    payload: str

class PacketWithExtra(Packet):
    extra: str

p = PacketWithExtra(type='hello', payload='v1', extra='english')
p2: Packet = p
print(p2)

Above, the value p2 is clearly both treated as a Packet and has an extra key that Packet does not define (extra).

@davidfstr davidfstr added the enhancement Enhancement to existing feature label Jul 28, 2023
@davidfstr
Copy link
Owner

I'm about to be on vacation for a week so I won't have any bandwidth to work on this feature myself in the short term.

@jaredculp if you (or anyone else) is interested in making a PR, I may be able to get it merged more quickly (before I leave on Saturday).

@jaredculp
Copy link
Author

@davidfstr happy to try!

Can't get a working dev environment set up on my mac (also tried docker):

 > [5/6] RUN poetry install:
#9 3.106       145│
#9 3.106       146│             if error is not None:
#9 3.106     → 147│                 raise error from None
#9 3.106       148│
#9 3.106       149│             return path
#9 3.106       150│
#9 3.106       151│     def _prepare_sdist(self, archive: Path, destination: Path | None = None) -> Path:
#9 3.106
#9 3.106 Note: This error originates from the build backend, and is likely not a problem with poetry but with pyyaml (6.0) not supporting PEP 517 builds. You can verify this by running 'pip wheel --use-pep517
"pyyaml (==6.0)"'.

@davidfstr
Copy link
Owner

davidfstr commented Jul 28, 2023

AttributeError: cython sources


"pyyaml (==6.0)"'.

You're probably running into this issue where Cython 3 broke the build process of PyYAML 5.x

Try this:

poetry shell
pip3 install "Cython<3.0" "pyyaml<6" --no-build-isolation
poetry install

@davidfstr davidfstr self-assigned this Nov 11, 2023
@davidfstr
Copy link
Owner

The ability to parse JSON with extra keys that are not in the original TypedDict definition will be in trycast 1.1.0, when it is released, probably in a day or two.

@davidfstr davidfstr added this to the v.1.1 milestone Nov 11, 2023
davidfstr added a commit that referenced this issue Apr 5, 2024
In particular:
* Tests: Fix to recognize TypedDict values with extra keys
    - References #19
davidfstr added a commit that referenced this issue Apr 5, 2024
In particular:
* Tests: Fix to recognize TypedDict values with extra keys
    - References #19
davidfstr added a commit that referenced this issue Apr 5, 2024
In particular:
* Tests: Fix to recognize TypedDict values with extra keys
    - References #19
davidfstr added a commit that referenced this issue Apr 5, 2024
In particular:
* Tests: Fix to recognize TypedDict values with extra keys
    - References #19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement Enhancement to existing feature
Projects
None yet
Development

No branches or pull requests

2 participants