Convert python objects to CLR objects dynamically

[yagweb]

Environment

• Pythonnet version:
• Python version: all
• Operating System:

Details

• Convert python objects to CLR objects is a basic need when embedding python. It can be done with the 'AsManagedObject' method if the target CLR type is known in the compiling time, but now I need to it dynamically.

  For example, suppose a python function returns a list, in which a element can be of type bool, int, str, list, dict etc. I want to convert the list to a List<object> in CLR. I write a class for this and it works well, which used a Dictionary<IntPtr, Func<PyObject, object>>.

  However, the code looks ugly, because the python types in Runtime class, such as PyIntType, are internal, so I use 'PythonEngine.Eval("int").Handle' to get the type instead. (XDecref should be called for the type as key)

  My question is, is there a better way to implement the dynamically converting? if not,
  could pythonnet expose this types for users by some way, such as a method with signature 'IntPtr GetPythonType(string type))'?

[den-run-ai]

@yagweb can you share full code example? I'm not sure I fully understand.

[yagweb]

@denfromufa

Suppose a Python function,

def test_py():
    return [1, "string", False]

In C#, I want to call the function and convert the result to a CLR List<object>

var res = test_py() //call the python function some way
//now, I want to convert res into List<object>
var converter = NewConverter()  //create a instance of PyConverter
var b = converter.Convert(res)    //b is a List of CLR objects

The PyConverter looks like,

public class PyConverter
    {
        public PyConverter()
        {
            this.Converters = new Dictionary<IntPtr, Func<PyObject, object>>();
        }

        private Dictionary<IntPtr, Func<PyObject, object>> Converters;
        
        public void Add(IntPtr type, Func<PyObject, object> func)
        {
            this.Converters.Add(type, func);
        }

        public object Convert(PyObject obj)
        {
            if(obj == null)
            {
                return null;
            }
            PyObject type = obj.GetPythonType();
            Func<PyObject, object> func;
            var state = Converters.TryGetValue(type.Handle, out func);
            if(!state)
            {
                throw new Exception($"Type {type.ToString()} not recognized");
            }
            return func(obj);
        }
    }

Here is the code how I use the PyConverter,

public PyConverter NewConverter()
{
           var converter = new PyConverter();
            using (Py.GIL())
            {
                //XIncref is needed, or keep the PyObject
                Converter.Add(PythonEngine.Eval("int").Handle, (args) => { return args.As<int>(); });
                Converter.Add(PythonEngine.Eval("str").Handle, (args) => { return args.As<string>(); });
                Converter.Add(PythonEngine.Eval("float").Handle, (args) => { return args.As<double>(); });
                Converter.Add(PythonEngine.Eval("bool").Handle, (args) => { return args.As<bool>(); });
                //Converter for list, omit here
            }
           return converter;
}

I want to find a better alternative for 'PythonEngine.Eval("int").Handle'. For example use the types in pythonnet, such as 'Runtime.PyIntType', but they are all internel.

and what about adding this PyConverter function in pythonnet?

[den-run-ai]

This looks a little complicated API. Why not use PyInt and its equivalents for other types?

https://github.com/pythonnet/pythonnet/blob/master/src/runtime/pyint.cs

[yagweb]

@denfromufa The example below shows that the element we get from a List (or Dict, Class attributes etc.) is wrapped in a PyObject class. (Or am I wrong?) So, PyInt etc. work only when we know the data type of the python object in advance, then we can call it in our code.

using (Py.GIL())
{
    var scope = Py.CreateScope();
    scope.Exec(
         "a=[1, \"2\"]"
    );
    dynamic a = scope.Get("a");
    object cc = a[0];
    Console.WriteLine(cc.GetType()); //print PyObject
    Console.WriteLine(cc.GetType() == typeof(PyInt)); //print false
    Console.WriteLine(cc);            
    scope.Dispose();
}

The problem here is we don't know the data type until runtime, or we want to have a single general converter to convert all these python objects into .NET object dynamically, instead writing lots of static convert functions.

In our .NET program, we can

1. Define Converters, register Convert functions for any class, include tuple, list, dict, ordereddict etc.
   Converters is stateless, so it can be defined once and used anywhere.
2. Call python functions
3. Use the Converters to convert PyObject into .NET object in a single line.

[yagweb]

This topic was merged into #623