Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Missing support for TypedDict #391

Open
Avasam opened this issue Mar 12, 2022 · 4 comments
Open

Missing support for TypedDict #391

Avasam opened this issue Mar 12, 2022 · 4 comments
Labels
needs triage type: feature A self-contained enhancement or new feature

Comments

@Avasam
Copy link

Avasam commented Mar 12, 2022

I cannot pass a TypedDict to the parameter _dict and if one of my properties is a TypedDict, it'll be saved as "TypedDict()"

In fact, any class that implements keys() or items() should probably work out of the box.

@pradyunsg
Copy link
Collaborator

Do you have a reproducer for this?

@Avasam
Copy link
Author

Avasam commented Mar 13, 2022

I could've sworn it didn't work... maybe I had accidentally left it as a dataclass... despite double checking.
Oh, or maybe I didn't clear my config so it kept loading then saving that same bad value from when it was a dataclass. My bad. Out of the box dataclass support would still be nice. But if not I can call asdict myself. Feel free to close this issue or I can change the title.

from dataclasses import dataclass, asdict
from typing import TypedDict
from toml import dump


@dataclass
class DataClass():
    def __init__(self, foo: str):
        self.foo = foo
    foo: str


class Parent(TypedDict):
    dictionary: dict[str, str]
    dataclass: DataClass
    asdict: dict[str, str]


dict_to_save = Parent(dataclass=DataClass(foo='bar'), asdict=asdict(DataClass(foo='bar')), dictionary={'foo': 'bar'})
print(dict_to_save)

with open("example.toml", "w", encoding="utf-8") as file:
    dump(dict_to_save, file)
dataclass = "DataClass(foo='bar')"

[asdict]
foo = "bar"

[dictionary]
foo = "bar"

@Avasam
Copy link
Author

Avasam commented Mar 13, 2022

It's still true that I can't use a TypedDict as my generic type though (passed to _dict)
image
image

@pradyunsg pradyunsg added needs triage type: feature A self-contained enhancement or new feature labels Apr 20, 2022
@pkulev
Copy link

pkulev commented Apr 29, 2022

I think it's better to use pydantic on top of the toml for data validation and proper deserialisation. In the following example I use my fork of the toml library with pathlib support, both for load()/dump() functions and native encoding and some other things.
Also my opinion now is that pydantic models are better than plain dataclasses when working with external sources (DB, user input via CLI and API, files like TOML and JSON).

As example I built simple ssh config "replacement" to show models a little.
Models represent configuration data, controller is responsible for file manipulations and other things, models should have no side effects as possible.

from ipaddress import IPv4Address
from pathlib import Path
from typing import Optional

import toml
from pydantic import BaseModel


class RemoteHost(BaseModel):
    user: Optional[str]
    host: IPv4Address


class SSHConfigSettings(BaseModel):
    x11_forwarding: Optional[bool] = False
    keepalive_timeout: Optional[int] = 60


class SSHConfig(BaseModel):
    hosts: list[RemoteHost]
    settings: SSHConfigSettings



class SSHConfigController:

    file = Path("/Users/most/.ssh/superconfig.toml")

    @classmethod
    def load(cls) -> SSHConfig:
        """Return validated and deserialized data."""

        raw = toml.load(cls.file)
        config = SSHConfig.parse_obj(raw)

        return config

    @classmethod
    def dump(cls, config: SSHConfig):
        """Dump validated and serialized as dict data."""

        contents = SSHConfig.validate(config).dict()
        toml.dump(contents, cls.file)


print("Creating stub config with some entries:\n")
config = SSHConfig(
    hosts=[
        RemoteHost(user="username", host="1.1.1.{0}".format(idx))
        for idx in range(2)
    ],
    settings=SSHConfigSettings(),  # defaults
)
print(config)

print("\nDumping into file...")
controller = SSHConfigController()
controller.dump(config)
print(f"{controller.file} contents:\n{controller.file.read_text()}")

print("\nRestoring config object from file...")
config = controller.load()
print(f"Restored config:\n{config}")

Output:

 ✗ python example.py
Creating stub config with some entries:

hosts=[RemoteHost(user='username', host=IPv4Address('1.1.1.0')), RemoteHost(user='username', host=IPv4Address('1.1.1.1'))] settings=SSHConfigSettings(x11_forwarding=False, keepalive_timeout=60)

Dumping into file...
/Users/most/.ssh/superconfig.toml contents:
[[hosts]]
user = "username"
host = "1.1.1.0"

[[hosts]]
user = "username"
host = "1.1.1.1"

[settings]
x11_forwarding = false
keepalive_timeout = 60
Restoring config object from file...
Restored config:
hosts=[RemoteHost(user='username', host=IPv4Address('1.1.1.0')), RemoteHost(user='username', host=IPv4Address('1.1.1.1'))] settings=SSHConfigSettings(x11_forwarding=False, keepalive_timeout=60)

So concluding I think that the toml is great toml library, but not very good at containing and restoring schema, this is another big complex topic and maybe it's better to use libraries that were designed to handle this. Schema, validation and ser/de to and from dict are handled by the pydantic, dumping/loading raw dict is handled by the toml. I did that in quite big CLI application, that used multiple TOML files as a database, it works so nice!

Please correct me if I'm wrong.

P.S.: the dataclass generates __init__ itself, so you can just:

@dataclass
class DataClass:
    foo: str

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
needs triage type: feature A self-contained enhancement or new feature
Projects
None yet
Development

No branches or pull requests

3 participants