Transducer error in GetFilteredScoreOp

[albertz]

OpCodeCompiler call: /usr/bin/g++-7 -shared -O2 -std=c++11 -I /home/kshur/.local/lib/python3.6/site-packages/tensorflow/include -I /home/kshur/.local/lib/python3.6/site-packages/tensorflow/include/external/nsync/public -fPIC -v -D_GLIBCXX_USE_CXX11_ABI=0 -DNDEBUG=1 -g /tmp/kshur/returnn_tf_cache/ops/GetFilteredScore/42f7cd62d5/GetFilteredScore.cc -o /tmp/kshur/returnn_tf_cache/ops/GetFilteredScore/42f7cd62d5/GetFilteredScore.so -L/home/kshur/.local/lib/python3.6/site-packages/tensorflow -l:libtensorflow_framework.so.2
OpCodeCompiler: /usr/bin/g++-7 failed.
Original stdout/stderr:
Using built-in specs.
COLLECT_GCC=/usr/bin/g++-7
COLLECT_LTO_WRAPPER=/usr/lib/gcc/x86_64-linux-gnu/7/lto-wrapper
OFFLOAD_TARGET_NAMES=nvptx-none
OFFLOAD_TARGET_DEFAULT=1
Target: x86_64-linux-gnu
Configured with: ../src/configure -v --with-pkgversion='Ubuntu 7.5.0-3ubuntu1~18.04' --with-bugurl=file:///usr/share/doc/gcc-7/README.Bugs --enable-languages=c,ada,c++,go,brig,d,fortran,objc,obj-c++ --prefix=/usr --with-gcc-major-version-only --program-suffix=-7 --program-prefix=x86_64-linux-gnu- --enable-shared --enable-linker-build-id --libexecdir=/usr/lib --without-included-gettext --enable-threads=posix --libdir=/usr/lib --enable-nls --enable-bootstrap --enable-clocale=gnu --enable-libstdcxx-debug --enable-libstdcxx-time=yes --with-default-libstdcxx-abi=new --enable-gnu-unique-object --disable-vtable-verify --enable-libmpx --enable-plugin --enable-default-pie --with-system-zlib --with-target-system-zlib --enable-objc-gc=auto --enable-multiarch --disable-werror --with-arch-32=i686 --with-abi=m64 --with-multilib-list=m32,m64,mx32 --enable-multilib --with-tune=generic --enable-offload-targets=nvptx-none --without-cuda-driver --enable-checking=release --build=x86_64-linux-gnu --host=x86_64-linux-gnu --target=x86_64-linux-gnu
Thread model: posix
gcc version 7.5.0 (Ubuntu 7.5.0-3ubuntu1~18.04)
COLLECT_GCC_OPTIONS='-shared' '-O2' '-std=c++11' '-I' '/home/kshur/.local/lib/python3.6/site-packages/tensorflow/include' '-I' '/home/kshur/.local/lib/python3.6/site-packages/tensorflow/include/external/nsync/public' '-fPIC' '-v' '-D' '_GLIBCXX_USE_CXX11_ABI=0' '-D' 'NDEBUG=1' '-g' '-o' '/tmp/kshur/returnn_tf_cache/ops/GetFilteredScore/42f7cd62d5/GetFilteredScore.so' '-L/home/kshur/.local/lib/python3.6/site-packages/tensorflow' '-shared-libgcc' '-mtune=generic' '-march=x86-64'
 /usr/lib/gcc/x86_64-linux-gnu/7/cc1plus -quiet -v -I /home/kshur/.local/lib/python3.6/site-packages/tensorflow/include -I /home/kshur/.local/lib/python3.6/site-packages/tensorflow/include/external/nsync/public -imultiarch x86_64-linux-gnu -D_GNU_SOURCE -D _GLIBCXX_USE_CXX11_ABI=0 -D NDEBUG=1 /tmp/kshur/returnn_tf_cache/ops/GetFilteredScore/42f7cd62d5/GetFilteredScore.cc -quiet -dumpbase GetFilteredScore.cc -mtune=generic -march=x86-64 -auxbase GetFilteredScore -g -O2 -std=c++11 -version -fPIC -fstack-protector-strong -Wformat -Wformat-security -o /tmp/cc9rtWst.s
GNU C++11 (Ubuntu 7.5.0-3ubuntu1~18.04) version 7.5.0 (x86_64-linux-gnu)
	compiled by GNU C version 7.5.0, GMP version 6.1.2, MPFR version 4.0.1, MPC version 1.1.0, isl version isl-0.19-GMP

GGC heuristics: --param ggc-min-expand=100 --param ggc-min-heapsize=131072
ignoring duplicate directory "/usr/include/x86_64-linux-gnu/c++/7"
ignoring nonexistent directory "/usr/local/include/x86_64-linux-gnu"
ignoring nonexistent directory "/usr/lib/gcc/x86_64-linux-gnu/7/../../../../x86_64-linux-gnu/include"
#include "..." search starts here:
#include <...> search starts here:
 /home/kshur/.local/lib/python3.6/site-packages/tensorflow/include
 /home/kshur/.local/lib/python3.6/site-packages/tensorflow/include/external/nsync/public
 /usr/include/c++/7
 /usr/include/x86_64-linux-gnu/c++/7
 /usr/include/c++/7/backward
 /usr/lib/gcc/x86_64-linux-gnu/7/include
 /usr/local/include
 /usr/lib/gcc/x86_64-linux-gnu/7/include-fixed
 /usr/include/x86_64-linux-gnu
 /usr/include
End of search list.
GNU C++11 (Ubuntu 7.5.0-3ubuntu1~18.04) version 7.5.0 (x86_64-linux-gnu)
	compiled by GNU C version 7.5.0, GMP version 6.1.2, MPFR version 4.0.1, MPC version 1.1.0, isl version isl-0.19-GMP

GGC heuristics: --param ggc-min-expand=100 --param ggc-min-heapsize=131072
Compiler executable checksum: 3eb3dc290cd5714c3e1c3ae751116f07
In file included from /home/kshur/.local/lib/python3.6/site-packages/tensorflow/include/tensorflow/core/framework/tensor.h:26:0,
                 from /home/kshur/.local/lib/python3.6/site-packages/tensorflow/include/tensorflow/core/framework/device_base.h:26,
                 from /home/kshur/.local/lib/python3.6/site-packages/tensorflow/include/tensorflow/core/framework/op_kernel.h:27,
                 from /tmp/kshur/returnn_tf_cache/ops/GetFilteredScore/42f7cd62d5/GetFilteredScore.cc:3:
/home/kshur/.local/lib/python3.6/site-packages/tensorflow/include/tensorflow/core/framework/types.h: In instantiation of 'struct tensorflow::DataTypeToEnum<std::basic_string<char> >':
/home/kshur/.local/lib/python3.6/site-packages/tensorflow/include/tensorflow/core/framework/tensor.h:734:45:   required from 'typename tensorflow::TTypes<T, NDIMS>::ConstTensor tensorflow::Tensor::tensor() const [with T = std::basic_string<char>; long unsigned int NDIMS = 2; typename tensorflow::TTypes<T, NDIMS>::ConstTensor = Eigen::TensorMap<Eigen::Tensor<const std::basic_string<char>, 2, 1, long int>, 16, Eigen::MakePointer>]'
/tmp/kshur/returnn_tf_cache/ops/GetFilteredScore/42f7cd62d5/GetFilteredScore.cc:51:70:   required from here
/home/kshur/.local/lib/python3.6/site-packages/tensorflow/include/tensorflow/core/framework/types.h:355:3: error: static assertion failed: Specified Data Type not supported
   static_assert(IsValidDataType<T>::value, "Specified Data Type not supported");
   ^~~~~~~~~~~~~
In file included from /home/kshur/.local/lib/python3.6/site-packages/tensorflow/include/tensorflow/core/framework/device_base.h:26:0,
                 from /home/kshur/.local/lib/python3.6/site-packages/tensorflow/include/tensorflow/core/framework/op_kernel.h:27,
                 from /tmp/kshur/returnn_tf_cache/ops/GetFilteredScore/42f7cd62d5/GetFilteredScore.cc:3:
/home/kshur/.local/lib/python3.6/site-packages/tensorflow/include/tensorflow/core/framework/tensor.h: In instantiation of 'typename tensorflow::TTypes<T, NDIMS>::ConstTensor tensorflow::Tensor::tensor() const [with T = std::basic_string<char>; long unsigned int NDIMS = 2; typename tensorflow::TTypes<T, NDIMS>::ConstTensor = Eigen::TensorMap<Eigen::Tensor<const std::basic_string<char>, 2, 1, long int>, 16, Eigen::MakePointer>]':
/tmp/kshur/returnn_tf_cache/ops/GetFilteredScore/42f7cd62d5/GetFilteredScore.cc:51:70:   required from here
/home/kshur/.local/lib/python3.6/site-packages/tensorflow/include/tensorflow/core/framework/tensor.h:734:45: error: 'v' is not a member of 'tensorflow::DataTypeToEnum<std::basic_string<char> >'
   CheckTypeAndIsAligned(DataTypeToEnum<T>::v());
                         ~~~~~~~~~~~~~~~~~~~~^~

[albertz]

I'm just posting this because I remember that I have seen this before, and it was already fixed, but probably not in all configs. So this is to make it easily searchable.

[albertz]

This probably happens with this config.

[albertz]

An updated version of this code (the whole function get_filtered_score_op, together with get_filtered_score_cpp and targetb_recomb_recog) can be found here (before it was moved over to returnn-common, so this should have all the Git edit history).

[gansaikhanshur]

Changing the functions fixes the above issue. I think this issue is fixed now and can be closed! Thank you a lot!