Failing to run infer.NET via pythonnet in a mono Docker container with JupyterLab

[dayalstrub-cma]

Outline

I would very much like to try to use Infer.NET and to read through the MBML book, but would like to do so by integrating Infer.NET into my usual Python/Jupyter workflow, thus pretending that Infer.NET is "just" a PP DSL, and ignoring the fact that it's actually C#, which I don't know and should maybe learn. I have been inspired by the recent write up of the tutorials using pythonnet, and think I'm not the only one that would want to try out Infer.NET this way.

Along the lines of what Tobimaru did a while back (see here), I am trying to build a Dockerfile using the mono image and to run the tutorials, but am getting an error that I don't understand. NOTE: Tobimaru's Dockerfile also throws up the same error.

I'm sorry if this is a silly .NET error rather than an Infer.NET one, but I couldn't figure it out simply searching online given my lack of .NET/C# understanding.

Details

Dockerfile

FROM mono:6.10

RUN apt-get update && apt-get install -y \
    python3-pip \
    clang \
    graphviz

# install infer.net packages and copy all dlls to same folder (so pythonnet can find them)
RUN mkdir -p /root/dotNet/packages && cd /root/dotNet/packages \
    && export NUGET="nuget install -ExcludeVersion" \
    && ${NUGET} Microsoft.ML.Probabilistic \
    && ${NUGET} Microsoft.ML.Probabilistic.Compiler \
    && mkdir -p /root/dotNet/libs \
    && find /root/dotNet/packages/ -type f -name '*.dll' -exec cp -n {} /root/dotNet/libs ';'

RUN pip3 install pycparser jupyterlab
RUN pip3 install pythonnet

WORKDIR /root/dev

ENTRYPOINT ["jupyter", "lab", "--no-browser", "--allow-root", "--ip=0.0.0.0", "--port=8888"]

Infer.Net two-coins example in Python run in Jupyter notebook

import sys
sys.path.append(r"../dotNet/libs")

import clr
clr.AddReference("Microsoft.ML.Probabilistic")
clr.AddReference("Microsoft.ML.Probabilistic.Compiler")

# from System import Boolean, Double, Int32, Array

from Microsoft.ML.Probabilistic import *
from Microsoft.ML.Probabilistic.Distributions import Bernoulli
from Microsoft.ML.Probabilistic.Models import Variable, InferenceEngine

firstCoin = Variable.Bernoulli(0.5)
secondCoin = Variable.Bernoulli(0.5)
bothHeads = firstCoin.op_BitwiseAnd(firstCoin, secondCoin)
engine = InferenceEngine()
print(f"Probability both coins are heads: {engine.Infer(bothHeads)}")
bothHeads.ObservedValue = False
print(f"Probability distribution over firstCoin: {engine.Infer(firstCoin)}")

Error message triggered by engine.Infer(bothHeads)

MissingMethodException: Method not found: void Microsoft.CodeAnalysis.Emit.EmitOptions..ctor(bool,Microsoft.CodeAnalysis.Emit.DebugInformationFormat,string,string,int,ulong,bool,Microsoft.CodeAnalysis.SubsystemVersion,string,bool,bool,System.Collections.Immutable.ImmutableArray`1<Microsoft.CodeAnalysis.Emit.InstrumentationKind>,System.Nullable`1<System.Security.Cryptography.HashAlgorithmName>)
  at Microsoft.ML.Probabilistic.Compiler.CodeCompiler.Compile (System.Collections.Generic.List`1[T] filenames, System.Collections.Generic.List`1[T] sources, System.Collections.Generic.ICollection`1[T] referencedAssemblies) [0x00074] in <ec23e96f4c7e4ba3ac56b5a8cd7034bf>:0 
  at Microsoft.ML.Probabilistic.Compiler.CodeCompiler.WriteAndCompile (System.Collections.Generic.List`1[T] typeDeclarations) [0x00011] in <ec23e96f4c7e4ba3ac56b5a8cd7034bf>:0 
  at Microsoft.ML.Probabilistic.Compiler.ModelCompiler.CompileWithoutParams[T] (System.Collections.Generic.List`1[T] itds) [0x00063] in <ec23e96f4c7e4ba3ac56b5a8cd7034bf>:0 
  at Microsoft.ML.Probabilistic.Compiler.ModelCompiler.CompileWithoutParams (Microsoft.ML.Probabilistic.Compiler.CodeModel.ITypeDeclaration itd, System.Reflection.MethodBase method, Microsoft.ML.Probabilistic.Compiler.AttributeRegistry`2[TObject,TAttribute] inputAttributes) [0x00016] in <ec23e96f4c7e4ba3ac56b5a8cd7034bf>:0 
  at Microsoft.ML.Probabilistic.Models.InferenceEngine.Compile () [0x000e0] in <ec23e96f4c7e4ba3ac56b5a8cd7034bf>:0 
  at Microsoft.ML.Probabilistic.Models.InferenceEngine.BuildAndCompile (System.Boolean inferOnlySpecifiedVars, System.Collections.Generic.IEnumerable`1[T] vars) [0x000e7] in <ec23e96f4c7e4ba3ac56b5a8cd7034bf>:0 
  at Microsoft.ML.Probabilistic.Models.InferenceEngine.GetCompiledInferenceAlgorithm (System.Boolean inferOnlySpecifiedVars, Microsoft.ML.Probabilistic.Models.IVariable var) [0x00017] in <ec23e96f4c7e4ba3ac56b5a8cd7034bf>:0 
  at Microsoft.ML.Probabilistic.Models.InferenceEngine.InferAll (System.Boolean inferOnlySpecifiedVars, Microsoft.ML.Probabilistic.Models.IVariable var) [0x00000] in <ec23e96f4c7e4ba3ac56b5a8cd7034bf>:0 
  at Microsoft.ML.Probabilistic.Models.InferenceEngine.Infer (Microsoft.ML.Probabilistic.Models.IVariable var) [0x00000] in <ec23e96f4c7e4ba3ac56b5a8cd7034bf>:0 
  at (wrapper managed-to-native) System.Reflection.RuntimeMethodInfo.InternalInvoke(System.Reflection.RuntimeMethodInfo,object,object[],System.Exception&)
  at System.Reflection.RuntimeMethodInfo.Invoke (System.Object obj, System.Reflection.BindingFlags invokeAttr, System.Reflection.Binder binder, System.Object[] parameters, System.Globalization.CultureInfo culture) [0x0006a] in <a17fa1457c5d44f2885ac746c1764ea5>:0 

[tminka]

Try setting

engine.Compiler.CompilerChoice = CompilerChoice.CodeDom

Note the Python.NET page says "the Mono runtime is less supported".

[DayalStrub]

Thanks! Does indeed work.

I hadn't paid enough attention to that warning, given pythonnet defaults to (and installs, in the case of the conda version) mono for linux. My bad.

A while back, I set up .NET kernels in Jupyter using .NET interactive and .NET Core on linux/Docker, so I could try ditch mono, but I think I will try crack on with learning/using Infer.NET instead using your fix, which sounds more fun.

Happy to close the issue.

[DayalStrub]

For anyone that stumbles upon this issue in the future, oddly enough installing pythonnet via conda also solves the issue, i.e. no need to set CompilerChoice.

Conda deals with dependencies itself to the extent of installing Mono (at least in linux, as I was building off the Jupyter Docker Stack that uses Ubuntu).