Dict keys / values are returned as they are regardless of the field

[rakanalh]

Hello,

Upon investigating an issue of a dataclass which has a Dict field of custom types, i found out that marshmallow (in the version pinned by marshmallow_dataclasses, 2.19.2) returns the key/value pairs as they are regardless of what Field we set for them.

An example:

from dataclasses import dataclass
import marshmallow
from marshmallow_dataclass import class_schema, _native_to_marshmallow

from typing import NewType, Dict, Any

T_Signature = bytes
Signature = NewType("Signature", T_Signature)

EMPTY_SIGNATURE = Signature(bytes(65))

class BytesField(marshmallow.fields.Field):
    """ Used for `bytes` in the dataclass, serialize to hex encoding"""

    def _serialize(self, value: bytes, attr: Any, obj: Any) -> str:
        return to_hex(value)

    def _deserialize(self, value: str, attr: Any, data: Any) -> bytes:
        return to_bytes(hexstr=value)

_native_to_marshmallow[Signature] = BytesField


@dataclass
class A:
    signatures: Dict[Signature, Signature]


a = A(signatures={
    b'123': b'456',
})

schema = class_schema(A)
print(schema().dump(a))

Here, you can see i am doing the following:

1. Declaring a custom type Signature of type bytes
2. Adding the type to marshmallow_dataclass._native_to_marshmallow dictionary with a custom Field
3. The BytesField in this case would convert the bytes to a hex string and vice versa.
4. Dumping a sample object.

The sample object is dumped as it was provided

MarshalResult(data={'signatures': {b'123': b'456'}}, errors={})

to_hex and to_bytes aren't even declared in this file so that should have error'ed but it doesn't because marshmallow isn't using the field but rather returning the values as they are.

Here's the code in marshmallow:
https://github.com/marshmallow-code/marshmallow/blob/ac9ff95432f929fb0fa2394c29b94d1f804a1504/src/marshmallow/fields.py#L281-L298

And here's the code for Dict:
https://github.com/marshmallow-code/marshmallow/blob/ac9ff95432f929fb0fa2394c29b94d1f804a1504/src/marshmallow/fields.py#L1083-L1102

As you can see, it doesn't use the fields passed down to them in marshmallow_dataclass dict code:

marshmallow_dataclass/marshmallow_dataclass/__init__.py

Lines 289 to 295 in d52d94a

	elif origin in (dict, Dict):
	key_type, value_type = typing_inspect.get_args(typ, True)
	return marshmallow.fields.Dict(
	keys=field_for_schema(key_type),
	values=field_for_schema(value_type),
	**metadata
	)

My suggestion here, is to provide your own "Dict" field implementation which is able to use the provided fields to serialize/deserialize keys according to their pre-defined field types.

What do you think?

[rakanalh]

Might be worth noting that a suggestion could be to define my own custom schema for the above dataclass. To be honest, this is exactly what i am trying to avoid given that i would like to completely separate serialization from my object declaration. IOW, no custom definitions / marshmallow-related-code for fields in my dataclass. This is why i am going the other route where i use class_schema directly.

[rakanalh]

I just checked out the new version of marshmallow 3.0.0rc6... seems like the new version is going to contain a proper implementation for dictionary types.

This renders this issue invalid.