How to turn a normal python dict into a numba.typed.Dict?

[scirocc]

Like this:

from numba import njit, typeof, typed, types
d1 = typed.Dict.empty(types.int32,types.int32)
normal_dict={}
normal_dict[1]=[1]

And then make
d1=typecast（normal_dict）

[stuartarchibald]

Thanks for the query. There's no typecast and it's unlikely there ever will be as fundamentally the CPython dictionary and Numba typed dictionary internal data structures are different. It might be possible to have a copy constructor at some point, but for now the best way to do what you want is either construct the normal_dict as a typed.dict in the first place or just copy via a loop, i.e.

from numba import njit, typeof, typed, types

d1 = typed.Dict.empty(types.int32,types.int32)
normal_dict={}
normal_dict[1]=[1]

for k, v in normal_dict.items():
    d1[k] = v

(which also highlights an issue in the above code, d1 is typed with values of int32 type but normal_dict has values which as lists).

[scirocc]

Wish to help.I don't know how to commit.Here is my solution

def pyObjToNumbaObj(pyObj):
    if type(pyObj)==dict:
        pyDict=pyObj
        keys=list(pyDict.keys())
        values=list(pyDict.values())
        if type(keys[0]) == str:nbhKeytype = nb.types.string
        elif type(keys[0]) == int:nbhKeytype = nb.types.int32
        elif type(keys[0]) == float: nbhKeytype = nb.types.float32
        if type(values[0])==int:
            nbh=nb.typed.Dict.empty(nbhKeytype,nb.types.int32)
            for i,key in enumerate(keys):nbh[key]=values[i]
            return(nbh)
        elif type(values[0])==str:
            nbh=nb.typed.Dict.empty(nbhKeytype,nb.types.string)
            for i,key in enumerate(keys):nbh[key]=values[i]
            return(nbh)
        elif type(values[0]) == float:
            nbh=nb.typed.Dict.empty(nbhKeytype,nb.types.float32)
            for i,key in enumerate(keys):nbh[key]=values[i]
            return(nbh)
        elif type(values[0])==dict:
            for i,subDict in enumerate(values):
                subDict=pyObjToNumbaObj(subDict)
                if i==0:nbh=nb.typed.Dict.empty(nbhKeytype,nb.typeof(subDict))
                nbh[keys[i]]=subDict
            return(nbh)
        elif type(values[0])==list:
            for i, subList in enumerate(values):
                subList=pyObjToNumbaObj(subList)
                if i==0:nbh=nb.typed.Dict.empty(nbhKeytype,nb.typeof(subList))
                nbh[keys[i]] = subList
            return(nbh)
    elif type(pyObj)==list:
        pyList=pyObj
        data = pyList[0]
        if type(data) == int:
            nbs = nb.typed.List.empty_list(nb.types.int32)
            for data_ in pyList: nbs.append(data_)
            return (nbs)
        elif type(data) == str:
            nbs = nb.typed.List.empty_list(nb.types.string)
            for data_ in pyList: nbs.append(data_)
            return (nbs)
        elif type(data) == float:
            nbs = nb.typed.List.empty_list(nb.types.float32)
            for data_ in pyList: nbs.append(data_)
            return (nbs)
        elif type(data) == dict:
            for i,subDict in enumerate(pyList):
                subDict = pyObjToNumbaObj(subDict)
                if i == 0: nbs = nb.typed.List.empty_list(nb.typeof(subDict))  
                nbs.append(subDict)
            return(nbs)
        elif type(data) == list:
            for i,subList in enumerate(pyList):
                subList=pyObjToNumbaObj(subList)
                if i==0: nbs = nb.typed.List.empty_list(nb.typeof(subList))  
                nbs.append(subList)
            return(nbs)

[github-actions]

This issue is marked as stale as it has had no activity in the past 30 days. Please close this issue if no further response or action is needed. Otherwise, please respond with any updates and confirm that this issue still needs to be addressed.

[gmarkall]

@scirocc Thanks for sharing your solution - this may serve as a start/inspiration for implementing a copy constructor in Numba for creating a typed Dict from a Python dict (though note I have not reviewed the code).

I'm removing the question label to prevent this feature request going stale again.

[sgbaird]

@scirocc, that worked for my use case - a simple lookup dictionary of periodic elements ('H', 'C', 'Fe', etc.) and a scalar property. Thank you!

[sf8193]

hello, trying to get this function to work with a dictionary of dictionary of lists, Like this

 for position, exit_dates in map_of_positions.items():
        sorted_exit_dates = sorted(exit_dates.items(), key=lambda x: x[0])
        sorted_map_of_positions[position] = dict(sorted_exit_dates)

    sorted_map = pyObjToNumbaObj(sorted_map_of_positions)

where position is a string and the types look like this

sorted_map = typed.Dict.empty(
      key_type=types.unicode_type,
      value_type=typed.Dict.empty(
          key_type=types.NPDatetime('D'),
          value_type=types.Tuple((types.float64[:],))
      )
  )

is there something i'm doing fundamentally wrong?

stack trace is

Traceback (most recent call last):
  File "main.py", line 370, in <module>
    sorted_map_of_positions = build_map_of_positions(df)
  File "main.py", line 224, in build_map_of_positions
    sorted_map = pyObjToNumbaObj(sorted_map_of_positions)
  File "main.py", line 167, in pyObjToNumbaObj
    subDict=pyObjToNumbaObj(subDict)
  File "main.py", line 175, in pyObjToNumbaObj
    if i==0:nbh=typed.Dict.empty(nbhKeytype,typeof(subList))
UnboundLocalError: local variable 'nbhKeytype' referenced before assignment

[gmarkall]

@sf8193 There's likely a bug in the pyObjToNumbaObj() function above, which will need tracking down and fixing - an UnboundLocalError shouldn't ever be thrown back to the callee of a function.

It would be better to move this discussion over to Discourse (https://numba.discourse.group) if further assistance is needed, as this isn't a Numba issue per se, and the original question was answered - the Numba Discourse is quite active and others might be able to suggest what the issue is and how to workaround it if you're struggling to fix up the implementation.