Serialising (to JSON) a custom datatype/structure

[chrisbeardy]

I have a PLC with OPC UA server, and I'm attempting to write something that will do a backup of all my variables as we have a lot of settings we don't want to accidentally lose (this particular PLC has a tendency to reset them all to default if you do a minor program change...).

I have some code that collects all the nodes then reads them (simplified here):

async def _export_all_data():
    data = await _get_all_data_from_plc()):
    _write_data_to_file(data)

async def _get_all_data_from_plc() -> [dict[str, Any]]:
    """Gets all the variable data from the PLC."""
    async with UaClient(load_settings().value.opc.url) as plc:
        await plc.load_data_type_definitions()
        variable_names: list[str] = []
        plc_node = plc.get_node("ns=3;s=DataBlocksGlobal")
        await _add_read_variables_recursive(plc, plc_node, variable_names)
        data = await plc.read_list_by_name(variable_names)  # this function is essentially just a wrapper for asyncio gather read node value
        return data

async def _add_read_variables_recursive(plc: UaClient, node: asyncua.common.Node, variables: list[str]):
    """Recursively browses through the root node and appends all found 'variable names' to the read list."""
    for child_node in await node.get_children():
        if await child_node.read_node_class() == ua.NodeClass.Object:
            await _add_read_variables_recursive(plc, child_node, variables)
        elif await child_node.read_node_class() == ua.NodeClass.Variable:
            access_level = await child_node.get_access_level()
            if not access_level:
                logger.warning(f"could not add {node} to read list due to access level")
                continue
            variables.append(child_node.nodeid.Identifier)


def _write_data_to_file(data: dict[str, Any]) -> Result:
    """Writes the data to a file in json format."""
    # can't serialize the custom types, write as string just so can see value? or custom parser?
    with open(Path("C:\\DataBackups\\test.json"), "w") as file:
        json.dump(data, file, indent=4)

This read results in a dictionary data which looks something like this:

data = {
    '"DB_OP1"."bVar"': False,
    '"DB_OP1"."iVar"': 12,
    '"DB_OP1"."ValueOfCustomStruct"': UDT_Custom(Var1='Hi', Var2='Bye'),
}

I rather naively thought I would see what would happen if I just tried to serialise this direct to JSON, but rather predictably I get the error:

TypeError: Object of type UDT_Custom is not JSON serializable

Has anyone tried anything similar, or have any ideas of how to potentially best serialise / export this data? It doesn't have to be JSON, just something fairly human-readable in case we need to put the data back in.

[AndreasHeine]

maybe its an class and not an object? methods in the class often trigger these kind of errors. dump it at the console or set a Breakpoint and check the object you try to serialize!

[chrisbeardy]

Yes, in this case UDT_Custom is a class autogenerated by the load_data_type_definitions(). However your hint helped me solve the issue, calling the dict on an object which is autogenerated by load_data_type_definitions() will actually work (with the current implementation).

    def to_dict(o):
        return o.__dict__

    try:
        with open(Path("C:\\DataBackups\\test.json"), "w") as file:
            json.dump(data, file, default=to_dict, indent=4)

It may be worth creating a helper function for this a bit like dataclasses.asdict() which means users don't have to call the magic method, this will also protect against internal changes to this class etc.

[schroeder-]

All no basic datatypes(int, float, str, ...) are based on Dataclasses. To serialize as json you need to use a custom encoder see this stackoverflow for example:
https://stackoverflow.com/questions/51286748/make-the-python-json-encoder-support-pythons-new-dataclasses

[chrisbeardy]

thanks, this works nicely!