[BUG] Pybind11 exception with tensorflow 2.2 env in python cpp communicatio

[spratik]

System information

• OS Platform and Distribution (e.g., Linux Ubuntu 16.04): window 10
• TensorFlow installed from (source or binary): pip install tensorflow-gpu==2.2
• TensorFlow version (use command below):2.2
• Python version:3.7.6
• CUDA/cuDNN version: 10, 10.1,8.0 / 7.6.5
• GPU model and memory: NVIDIA Geforce GTX 1050 Ti & 12 GB

Issue description

Describe the current behavior
Below code is working fine while running through tensorflow2.0 env but same code if I run through tensorflow2.2 env then it's give below exception at PyImport_Import(pName)

Error : Microsoft c++ exception : pybind11::error_already_set at memory location XXXXXX

Describe the expected behavior
should not through pybind11::error_already_set while just importing tensorflow library

Reproducible example code

C++ code

#define PY_SSIZE_T_CLEAN
#include <Python.h>
#include
using namespace std;
int main(int argc, char *argv[])
{
PyObject *pName, *pModule;
std::string pythonFile = "sample"
std::string tensoflow_env = "tensorflow_env22_path";
wchar *env = Py_DecodeLocale(tensoflow_env.c_str(),NULL);
Py_SetPythonHome(env);
Py_Initialize();
pName = PyUnicode_DecodeFSDefault(pythonFile.c_str());
pModule = PyImport_Import(pName);

Py_DECREF(pName);
Py_DECREF(pModule);

if (Py_FinalizeEx() < 0) {
return 120;
}
return 0;
}

Python Code :
Below is sample.py file for a reference.

import tensorflow as tf

Note that you have to set below path in system environment path variable. then only above code will work.
<Tensorflow_flow_env22>
<Tensorflow_flow_env22>\Scripts
<Tensorflow_flow_env22>\Library\bin

I have used Visual studio 2015 with v140 compiler.

[bstaletic]

Your example code doesn't even touch pybind11, therefore this can't be a pybind11 bug.

[spratik]

Your example code doesn't even touch pybind11, therefore this can't be a pybind11 bug.

Yes it is actually problem of pybind11. actually pybind11 use Cpython function only as per my analysis. below is import function of pybind11 internal function.

Below is source code with pybind11 import for your reference.
#define PY_SSIZE_T_CLEAN
#include <Python.h>
#include <pybind11\embed.h>
#include
using namespace std;
namespace py =pybind11;
int main(int argc, char *argv[])
{
PyObject *pName, *pModule;
std::string pythonFile = "sample"
std::string tensoflow_env = "tensorflow_env22_path";
wchar *env = Py_DecodeLocale(tensoflow_env.c_str(),NULL);
Py_SetPythonHome(env);
Py_Initialize();
pName = PyUnicode_DecodeFSDefault(pythonFile.c_str());
//pModule = PyImport_Import(pName);

try{
auto pymodle = py::model::import("sample")
}
catch(py::error_already_set &e) {

}

Py_DECREF(pName);
Py_DECREF(pModule);

if (Py_FinalizeEx() < 0) {
return 120;
}
return 0;
}

Python Code :
Below is sample.py file for a reference.

import tensorflow as tf

Note that getting pybind11::error_already_set at memory location XXXXX exception at below line.
auto pymodle = py::model::import("sample")

Unfortunately this exception is not even catch by using catch block as mention above code. Pybind11:: error_already_set exception thrown internally. means that If I find definition of py::model::import then it will call PYimport_ImportModule function internally and PYimport_ImportModule giving pybind11::error_already_set exception.
below screen short function call internally.

[virtuald]

@spratik if the exception is escaping the import then that sounds like tensorflow isn't using pybind11 correctly. File a bug with them.