New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Python 3.7 compatibility #20517

Closed
homofortis opened this Issue Jul 3, 2018 · 72 comments

Comments

Projects
None yet
@homofortis
Copy link

homofortis commented Jul 3, 2018

I'm sure developers are working hard to catch up with Python 3.7.
Is there any timeline?

pip3 install tensorflow - apparently does not work, building from source:

OS Platform and Distribution: Mac OS X 10.13.5
Python: Python 3.7.0 (Homebrew)
TensorFlow installed from: source (https://github.com/tensorflow/tensorflow.git)
TensorFlow version: TensorFlow 1.9.0-rc2
Bazel version:

Build label: 0.15.0-homebrew
Build target: bazel-out/darwin-opt/bin/src/main/java/com/google/devtools/build/lib/bazel/BazelServer_deploy.jar
Build time: Tue Jun 26 12:42:27 2018 (1530016947)
Build timestamp: 1530016947
Build timestamp as int: 1530016947

CUDA/cuDNN version: None
GPU model and memory: None
Exact command to reproduce:
bazel build --config=opt //tensorflow/tools/pip_package:build_pip_package

Starting local Bazel server and connecting to it...
...........................
WARNING: /private/var/tmp/_bazel_zardoz/5e080a8a46c0e2b2146c013eb1079337/external/grpc/BUILD:1992:1: in srcs attribute of cc_library rule @grpc//:grpc_nanopb: please do not import '@grpc//third_party/nanopb:pb_common.c' directly. You should either move the file to this package or depend on an appropriate rule there. Since this rule was created by the macro 'grpc_generate_one_off_targets', the error might have been caused by the macro implementation in /private/var/tmp/_bazel_zardoz/5e080a8a46c0e2b2146c013eb1079337/external/grpc/bazel/grpc_build_system.bzl:172:12
WARNING: /private/var/tmp/_bazel_zardoz/5e080a8a46c0e2b2146c013eb1079337/external/grpc/BUILD:1992:1: in srcs attribute of cc_library rule @grpc//:grpc_nanopb: please do not import '@grpc//third_party/nanopb:pb_decode.c' directly. You should either move the file to this package or depend on an appropriate rule there. Since this rule was created by the macro 'grpc_generate_one_off_targets', the error might have been caused by the macro implementation in /private/var/tmp/_bazel_zardoz/5e080a8a46c0e2b2146c013eb1079337/external/grpc/bazel/grpc_build_system.bzl:172:12
WARNING: /private/var/tmp/_bazel_zardoz/5e080a8a46c0e2b2146c013eb1079337/external/grpc/BUILD:1992:1: in srcs attribute of cc_library rule @grpc//:grpc_nanopb: please do not import '@grpc//third_party/nanopb:pb_encode.c' directly. You should either move the file to this package or depend on an appropriate rule there. Since this rule was created by the macro 'grpc_generate_one_off_targets', the error might have been caused by the macro implementation in /private/var/tmp/_bazel_zardoz/5e080a8a46c0e2b2146c013eb1079337/external/grpc/bazel/grpc_build_system.bzl:172:12
WARNING: /Users/zardoz/Projects/tensorflow/tensorflow/contrib/learn/BUILD:17:1: in py_library rule //tensorflow/contrib/learn:learn: target '//tensorflow/contrib/learn:learn' depends on deprecated target '//tensorflow/contrib/session_bundle:exporter': No longer supported. Switch to SavedModel immediately.
WARNING: /Users/zardoz/Projects/tensorflow/tensorflow/contrib/learn/BUILD:17:1: in py_library rule //tensorflow/contrib/learn:learn: target '//tensorflow/contrib/learn:learn' depends on deprecated target '//tensorflow/contrib/session_bundle:gc': No longer supported. Switch to SavedModel immediately.
WARNING: /Users/zardoz/Projects/tensorflow/tensorflow/contrib/timeseries/python/timeseries/BUILD:356:1: in py_library rule //tensorflow/contrib/timeseries/python/timeseries:ar_model: target '//tensorflow/contrib/timeseries/python/timeseries:ar_model' depends on deprecated target '//tensorflow/contrib/distributions:distributions_py': TensorFlow Distributions has migrated to TensorFlow Probability (https://github.com/tensorflow/probability). Deprecated copies remaining in tf.contrib.distributions are unmaintained, unsupported, and will be removed by late 2018. You should update all usage of `tf.contrib.distributions` to `tfp.distributions`.
WARNING: /Users/zardoz/Projects/tensorflow/tensorflow/contrib/timeseries/python/timeseries/state_space_models/BUILD:73:1: in py_library rule //tensorflow/contrib/timeseries/python/timeseries/state_space_models:kalman_filter: target '//tensorflow/contrib/timeseries/python/timeseries/state_space_models:kalman_filter' depends on deprecated target '//tensorflow/contrib/distributions:distributions_py': TensorFlow Distributions has migrated to TensorFlow Probability (https://github.com/tensorflow/probability). Deprecated copies remaining in tf.contrib.distributions are unmaintained, unsupported, and will be removed by late 2018. You should update all usage of `tf.contrib.distributions` to `tfp.distributions`.
WARNING: /Users/zardoz/Projects/tensorflow/tensorflow/contrib/timeseries/python/timeseries/state_space_models/BUILD:230:1: in py_library rule //tensorflow/contrib/timeseries/python/timeseries/state_space_models:filtering_postprocessor: target '//tensorflow/contrib/timeseries/python/timeseries/state_space_models:filtering_postprocessor' depends on deprecated target '//tensorflow/contrib/distributions:distributions_py': TensorFlow Distributions has migrated to TensorFlow Probability (https://github.com/tensorflow/probability). Deprecated copies remaining in tf.contrib.distributions are unmaintained, unsupported, and will be removed by late 2018. You should update all usage of `tf.contrib.distributions` to `tfp.distributions`.
WARNING: /Users/zardoz/Projects/tensorflow/tensorflow/contrib/bayesflow/BUILD:17:1: in py_library rule //tensorflow/contrib/bayesflow:bayesflow_py: target '//tensorflow/contrib/bayesflow:bayesflow_py' depends on deprecated target '//tensorflow/contrib/distributions:distributions_py': TensorFlow Distributions has migrated to TensorFlow Probability (https://github.com/tensorflow/probability). Deprecated copies remaining in tf.contrib.distributions are unmaintained, unsupported, and will be removed by late 2018. You should update all usage of `tf.contrib.distributions` to `tfp.distributions`.
WARNING: /Users/zardoz/Projects/tensorflow/tensorflow/contrib/seq2seq/BUILD:23:1: in py_library rule //tensorflow/contrib/seq2seq:seq2seq_py: target '//tensorflow/contrib/seq2seq:seq2seq_py' depends on deprecated target '//tensorflow/contrib/distributions:distributions_py': TensorFlow Distributions has migrated to TensorFlow Probability (https://github.com/tensorflow/probability). Deprecated copies remaining in tf.contrib.distributions are unmaintained, unsupported, and will be removed by late 2018. You should update all usage of `tf.contrib.distributions` to `tfp.distributions`.
WARNING: /Users/zardoz/Projects/tensorflow/tensorflow/contrib/kfac/python/ops/BUILD:80:1: in py_library rule //tensorflow/contrib/kfac/python/ops:loss_functions: target '//tensorflow/contrib/kfac/python/ops:loss_functions' depends on deprecated target '//tensorflow/contrib/distributions:distributions_py': TensorFlow Distributions has migrated to TensorFlow Probability (https://github.com/tensorflow/probability). Deprecated copies remaining in tf.contrib.distributions are unmaintained, unsupported, and will be removed by late 2018. You should update all usage of `tf.contrib.distributions` to `tfp.distributions`.
WARNING: /Users/zardoz/Projects/tensorflow/tensorflow/contrib/BUILD:14:1: in py_library rule //tensorflow/contrib:contrib_py: target '//tensorflow/contrib:contrib_py' depends on deprecated target '//tensorflow/contrib/distributions:distributions_py': TensorFlow Distributions has migrated to TensorFlow Probability (https://github.com/tensorflow/probability). Deprecated copies remaining in tf.contrib.distributions are unmaintained, unsupported, and will be removed by late 2018. You should update all usage of `tf.contrib.distributions` to `tfp.distributions`.
INFO: Analysed target //tensorflow/tools/pip_package:build_pip_package (303 packages loaded).
INFO: Found 1 target...
INFO: From Linking external/grpc/libgrpc_base_c.a [for host]:
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ranlib: file: bazel-out/host/bin/external/grpc/libgrpc_base_c.a(endpoint_pair_uv.o) has no symbols
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ranlib: file: bazel-out/host/bin/external/grpc/libgrpc_base_c.a(endpoint_pair_windows.o) has no symbols
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ranlib: file: bazel-out/host/bin/external/grpc/libgrpc_base_c.a(ev_windows.o) has no symbols
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ranlib: file: bazel-out/host/bin/external/grpc/libgrpc_base_c.a(fork_windows.o) has no symbols
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ranlib: file: bazel-out/host/bin/external/grpc/libgrpc_base_c.a(gethostname_fallback.o) has no symbols
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ranlib: file: bazel-out/host/bin/external/grpc/libgrpc_base_c.a(gethostname_host_name_max.o) has no symbols
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ranlib: file: bazel-out/host/bin/external/grpc/libgrpc_base_c.a(iocp_windows.o) has no symbols
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ranlib: file: bazel-out/host/bin/external/grpc/libgrpc_base_c.a(iomgr_windows.o) has no symbols
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ranlib: file: bazel-out/host/bin/external/grpc/libgrpc_base_c.a(pollset_set_windows.o) has no symbols
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ranlib: file: bazel-out/host/bin/external/grpc/libgrpc_base_c.a(pollset_uv.o) has no symbols
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ranlib: file: bazel-out/host/bin/external/grpc/libgrpc_base_c.a(pollset_windows.o) has no symbols
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ranlib: file: bazel-out/host/bin/external/grpc/libgrpc_base_c.a(resolve_address_windows.o) has no symbols
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ranlib: file: bazel-out/host/bin/external/grpc/libgrpc_base_c.a(socket_utils_linux.o) has no symbols
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ranlib: file: bazel-out/host/bin/external/grpc/libgrpc_base_c.a(socket_utils_windows.o) has no symbols
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ranlib: file: bazel-out/host/bin/external/grpc/libgrpc_base_c.a(socket_windows.o) has no symbols
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ranlib: file: bazel-out/host/bin/external/grpc/libgrpc_base_c.a(tcp_client_windows.o) has no symbols
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ranlib: file: bazel-out/host/bin/external/grpc/libgrpc_base_c.a(tcp_server_utils_posix_noifaddrs.o) has no symbols
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ranlib: file: bazel-out/host/bin/external/grpc/libgrpc_base_c.a(tcp_server_windows.o) has no symbols
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ranlib: file: bazel-out/host/bin/external/grpc/libgrpc_base_c.a(tcp_uv.o) has no symbols
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ranlib: file: bazel-out/host/bin/external/grpc/libgrpc_base_c.a(tcp_windows.o) has no symbols
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ranlib: file: bazel-out/host/bin/external/grpc/libgrpc_base_c.a(timer_uv.o) has no symbols
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ranlib: file: bazel-out/host/bin/external/grpc/libgrpc_base_c.a(unix_sockets_posix_noop.o) has no symbols
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ranlib: file: bazel-out/host/bin/external/grpc/libgrpc_base_c.a(wakeup_fd_eventfd.o) has no symbols
INFO: From Linking external/grpc/libalts_util.a [for host]:
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ranlib: file: bazel-out/host/bin/external/grpc/libalts_util.a(check_gcp_environment_linux.o) has no symbols
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ranlib: file: bazel-out/host/bin/external/grpc/libalts_util.a(check_gcp_environment_windows.o) has no symbols
INFO: From Linking external/grpc/libtsi.a [for host]:
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ranlib: file: bazel-out/host/bin/external/grpc/libtsi.a(ssl_session_openssl.o) has no symbols
INFO: From Linking external/grpc/libgrpc++_base.a [for host]:
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ranlib: file: bazel-out/host/bin/external/grpc/libgrpc++_base.a(rpc_method.o) has no symbols
INFO: From Linking external/grpc/libgpr_base.a [for host]:
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ranlib: file: bazel-out/host/bin/external/grpc/libgpr_base.a(cpu_iphone.o) has no symbols
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ranlib: file: bazel-out/host/bin/external/grpc/libgpr_base.a(cpu_linux.o) has no symbols
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ranlib: file: bazel-out/host/bin/external/grpc/libgpr_base.a(cpu_windows.o) has no symbols
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ranlib: file: bazel-out/host/bin/external/grpc/libgpr_base.a(env_linux.o) has no symbols
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ranlib: file: bazel-out/host/bin/external/grpc/libgpr_base.a(env_windows.o) has no symbols
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ranlib: file: bazel-out/host/bin/external/grpc/libgpr_base.a(log_android.o) has no symbols
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ranlib: file: bazel-out/host/bin/external/grpc/libgpr_base.a(log_linux.o) has no symbols
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ranlib: file: bazel-out/host/bin/external/grpc/libgpr_base.a(log_windows.o) has no symbols
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ranlib: file: bazel-out/host/bin/external/grpc/libgpr_base.a(string_util_windows.o) has no symbols
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ranlib: file: bazel-out/host/bin/external/grpc/libgpr_base.a(string_windows.o) has no symbols
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ranlib: file: bazel-out/host/bin/external/grpc/libgpr_base.a(sync_windows.o) has no symbols
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ranlib: file: bazel-out/host/bin/external/grpc/libgpr_base.a(time_windows.o) has no symbols
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ranlib: file: bazel-out/host/bin/external/grpc/libgpr_base.a(tls_pthread.o) has no symbols
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ranlib: file: bazel-out/host/bin/external/grpc/libgpr_base.a(tmpfile_msys.o) has no symbols
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ranlib: file: bazel-out/host/bin/external/grpc/libgpr_base.a(tmpfile_windows.o) has no symbols
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ranlib: file: bazel-out/host/bin/external/grpc/libgpr_base.a(wrap_memcpy.o) has no symbols
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ranlib: file: bazel-out/host/bin/external/grpc/libgpr_base.a(thd_windows.o) has no symbols
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ranlib: file: bazel-out/host/bin/external/grpc/libgpr_base.a(stap_timers.o) has no symbols
INFO: From Linking external/grpc/third_party/address_sorting/libaddress_sorting.a [for host]:
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ranlib: file: bazel-out/host/bin/external/grpc/third_party/address_sorting/libaddress_sorting.a(address_sorting_windows.o) has no symbols
ERROR: /Users/zardoz/Projects/tensorflow/tensorflow/python/BUILD:5315:1: Executing genrule //tensorflow/python:framework/fast_tensor_util.pyx_cython_translation failed (Exit 1)
Traceback (most recent call last):
  File "/private/var/tmp/_bazel_zardoz/5e080a8a46c0e2b2146c013eb1079337/execroot/org_tensorflow/bazel-out/host/bin/external/cython/cython_binary.runfiles/cython/cython.py", line 17, in <module>
    main(command_line = 1)
  File "/private/var/tmp/_bazel_zardoz/5e080a8a46c0e2b2146c013eb1079337/external/cython/Cython/Compiler/Main.py", line 720, in main
    result = compile(sources, options)
  File "/private/var/tmp/_bazel_zardoz/5e080a8a46c0e2b2146c013eb1079337/external/cython/Cython/Compiler/Main.py", line 695, in compile
    return compile_multiple(source, options)
  File "/private/var/tmp/_bazel_zardoz/5e080a8a46c0e2b2146c013eb1079337/external/cython/Cython/Compiler/Main.py", line 666, in compile_multiple
    context = options.create_context()
  File "/private/var/tmp/_bazel_zardoz/5e080a8a46c0e2b2146c013eb1079337/external/cython/Cython/Compiler/Main.py", line 590, in create_context
    self.cplus, self.language_level, options=self)
  File "/private/var/tmp/_bazel_zardoz/5e080a8a46c0e2b2146c013eb1079337/external/cython/Cython/Compiler/Main.py", line 75, in __init__
    from . import Builtin, CythonScope
  File "/private/var/tmp/_bazel_zardoz/5e080a8a46c0e2b2146c013eb1079337/external/cython/Cython/Compiler/CythonScope.py", line 5, in <module>
    from .UtilityCode import CythonUtilityCode
  File "/private/var/tmp/_bazel_zardoz/5e080a8a46c0e2b2146c013eb1079337/external/cython/Cython/Compiler/UtilityCode.py", line 3, in <module>
    from .TreeFragment import parse_from_strings, StringParseContext
  File "/private/var/tmp/_bazel_zardoz/5e080a8a46c0e2b2146c013eb1079337/external/cython/Cython/Compiler/TreeFragment.py", line 17, in <module>
    from .Visitor import VisitorTransform
  File "/private/var/tmp/_bazel_zardoz/5e080a8a46c0e2b2146c013eb1079337/external/cython/Cython/Compiler/Visitor.py", line 15, in <module>
    from . import ExprNodes
  File "/private/var/tmp/_bazel_zardoz/5e080a8a46c0e2b2146c013eb1079337/external/cython/Cython/Compiler/ExprNodes.py", line 2875
    await = None
          ^
SyntaxError: invalid syntax
Target //tensorflow/tools/pip_package:build_pip_package failed to build
Use --verbose_failures to see the command lines of failed build steps.
INFO: Elapsed time: 179.318s, Critical Path: 6.38s
INFO: 413 processes: 413 local.
FAILED: Build did NOT complete successfully

@homofortis homofortis changed the title Python 3.7 officailly released Python 3.7 officially released Jul 3, 2018

@homofortis homofortis changed the title Python 3.7 officially released Python 3.7 compatibility Jul 3, 2018

@homofortis homofortis closed this Jul 3, 2018

@homofortis homofortis reopened this Jul 9, 2018

@tensorflowbutler

This comment has been minimized.

Copy link
Member

tensorflowbutler commented Jul 9, 2018

Thank you for your post. We noticed you have not filled out the following field in the issue template. Could you update them if they are relevant in your case, or leave them as N/A? Thanks.
Have I written custom code
OS Platform and Distribution
TensorFlow installed from
TensorFlow version
Bazel version
CUDA/cuDNN version
GPU model and memory
Exact command to reproduce

@homofortis

This comment has been minimized.

Copy link
Author

homofortis commented Jul 9, 2018

Updated the original post as requested.

@salmanazarr

This comment has been minimized.

Copy link

salmanazarr commented Jul 9, 2018

@homofortis You could use this meanwhile with Homebrew to downgrade your Python version.

brew install https://raw.githubusercontent.com/Homebrew/homebrew-core/f2a764ef944b1080be64bd88dca9a1d80130c558/Formula/python.rb

async and await are now keywords and need to be replaced in the TF codebase. See https://docs.python.org/3/whatsnew/3.7.html#summary-release-highlights

@lutzroeder

This comment has been minimized.

Copy link

lutzroeder commented Jul 17, 2018

@homofortis

This comment has been minimized.

Copy link
Author

homofortis commented Jul 18, 2018

@activatedgeek Pardon me, but I fail to see how downgrading answers the the OP question regarding the timeline of making Tensorflow compatible with Python 3.7 (released almost a month ago).

@salmanazarr

This comment has been minimized.

Copy link

salmanazarr commented Jul 18, 2018

@homofortis Apologies. I probably missed a few words in there and thought that your main objective was to compile from source. A lot of searches were leading to this issue, I thought it would be good for everyone who's looking to just run Tensorflow.

@daa

This comment has been minimized.

Copy link

daa commented Jul 19, 2018

As I see from description diagnostics is not related to tensorflow compatibility with Python-3.7 but with usage of too old Cython and currently exactly this issue is not reproducing because Cython mentioned in Bazel workspace is new enough. On the other hand there are at least 2 Python-3.7 compatibility issues:

  • usage of async keyword as variable name in pywrap_tensorflow_internal.py generated from tensorflow/c/eager/c_api.{h,cc} - #20690
  • and breakage caused by change PyUnicode_AsUTF8AndSize()'s return type from char * to const char *
    Maybe it would be better to put the latter to separate issue.
@cgohlke

This comment has been minimized.

Copy link

cgohlke commented Jul 19, 2018

FWIW, I just built (not tested yet) tensorflow 1.9 with MKL on Windows for Python 3.7 using VS2017 and the following patch:

diff --git a/tensorflow/c/eager/c_api.h b/tensorflow/c/eager/c_api.h
index 1862af3ce2..093b97110f 100644
--- a/tensorflow/c/eager/c_api.h
+++ b/tensorflow/c/eager/c_api.h
@@ -76,7 +76,7 @@ typedef enum TFE_ContextDevicePlacementPolicy {
 // Sets the default execution mode (sync/async). Note that this can be
 // overridden per thread using TFE_ContextSetAsyncForThread.
 TF_CAPI_EXPORT extern void TFE_ContextOptionsSetAsync(TFE_ContextOptions*,
-                                                      unsigned char async);
+                                                      unsigned char is_async);
 
 TF_CAPI_EXPORT extern void TFE_ContextOptionsSetDevicePlacementPolicy(
     TFE_ContextOptions*, TFE_ContextDevicePlacementPolicy);
@@ -125,7 +125,7 @@ TFE_ContextGetDevicePlacementPolicy(TFE_Context*);
 
 // Overrides the execution mode (sync/async) for the current thread.
 TF_CAPI_EXPORT extern void TFE_ContextSetAsyncForThread(TFE_Context*,
-                                                        unsigned char async,
+                                                        unsigned char is_async,
                                                         TF_Status* status);
 
 // Causes the calling thread to block till all ops dispatched in async mode
diff --git a/tensorflow/core/platform/windows/port.cc b/tensorflow/core/platform/windows/port.cc
index 174f41a993..b06434620e 100644
--- a/tensorflow/core/platform/windows/port.cc
+++ b/tensorflow/core/platform/windows/port.cc
@@ -57,6 +57,11 @@ int NumSchedulableCPUs() {
   return system_info.dwNumberOfProcessors;
 }
 
+int NumHyperthreadsPerCore() {
+  static const int ht_per_core = tensorflow::port::CPUIDNumSMT();
+  return (ht_per_core > 0) ? ht_per_core : 1;
+}
+
 void* AlignedMalloc(size_t size, int minimum_alignment) {
 #ifdef TENSORFLOW_USE_JEMALLOC
   void* ptr = NULL;
diff --git a/tensorflow/python/eager/pywrap_tfe_src.cc b/tensorflow/python/eager/pywrap_tfe_src.cc
index 6c9481c3af..13edbb07db 100644
--- a/tensorflow/python/eager/pywrap_tfe_src.cc
+++ b/tensorflow/python/eager/pywrap_tfe_src.cc
@@ -813,7 +813,7 @@ char* TFE_GetPythonString(PyObject* o) {
   }
 #if PY_MAJOR_VERSION >= 3
   if (PyUnicode_Check(o)) {
-    return PyUnicode_AsUTF8(o);
+    return (char *)PyUnicode_AsUTF8(o);
   }
 #endif
   return nullptr;
diff --git a/tensorflow/python/lib/core/ndarray_tensor.cc b/tensorflow/python/lib/core/ndarray_tensor.cc
index 9df38d464c..4150fbfdd4 100644
--- a/tensorflow/python/lib/core/ndarray_tensor.cc
+++ b/tensorflow/python/lib/core/ndarray_tensor.cc
@@ -154,7 +154,7 @@ Status PyBytesArrayMap(PyArrayObject* array, F f) {
     if (PyUnicode_Check(item.get())) {
 #if PY_VERSION_HEX >= 0x03030000
       // Accept unicode by converting to UTF-8 bytes.
-      ptr = PyUnicode_AsUTF8AndSize(item.get(), &len);
+      ptr = (char *)PyUnicode_AsUTF8AndSize(item.get(), &len);
       if (!ptr) {
         return errors::Internal("Unable to get element as UTF-8.");
       }
diff --git a/tensorflow/python/lib/core/py_func.cc b/tensorflow/python/lib/core/py_func.cc
index 30c1a9c759..231a66de59 100644
--- a/tensorflow/python/lib/core/py_func.cc
+++ b/tensorflow/python/lib/core/py_func.cc
@@ -322,7 +322,7 @@ Status ConvertNdarrayToTensor(PyObject* obj, Tensor* ret) {
         Py_ssize_t el_size;
         if (PyBytes_AsStringAndSize(input_data[i], &el, &el_size) == -1) {
 #if PY_MAJOR_VERSION >= 3
-          el = PyUnicode_AsUTF8AndSize(input_data[i], &el_size);
+          el = (char *)PyUnicode_AsUTF8AndSize(input_data[i], &el_size);
 #else
           el = nullptr;
           if (PyUnicode_Check(input_data[i])) {
@serhiy-storchaka

This comment has been minimized.

Copy link

serhiy-storchaka commented Jul 31, 2018

I prefer to add the const qualifier to the target instead of removing it from from the result of PyUnicode_AsUTF8AndSize(). This is a constant string, and it should not be modified.

@cy89

This comment has been minimized.

Copy link

cy89 commented Aug 1, 2018

@asimshankar says that he's been reviewing and sending PRs on this topic. I'll assign him.

@cy89 cy89 assigned asimshankar and unassigned cy89 Aug 1, 2018

tensorflow-copybara pushed a commit that referenced this issue Aug 1, 2018

Python 3.7 Compatibility: Py_UnicodeAsUTF8AndSize() returns a "const …
…char*"

instead of a "char*".

(See https://docs.python.org/3/whatsnew/3.7.html#c-api-changes)

There are additional changes needed for Python 3.7 compatibility,
this change just pulls out one of them
(and subsumes a related attempt in #21202 and #20766)

Helps with #20517

PiperOrigin-RevId: 207008013
@rharish101

This comment has been minimized.

Copy link

rharish101 commented Aug 8, 2018

I'm facing similar issues when building from source too, using python 3.7 in Arch Linux.

ERROR: /home/rharish/.cache/bazel/_bazel_rharish/5d4d7b1255c710f6c814ab2f3f084405/external/protobuf_archive/BUILD:659:1: C++ compilation of rule '@protobuf_archive//:python/google/protobuf/pyext/_message.so' failed (Exit 1): crosstool_wrapper_driver_is_not_gcc failed: error executing command 
  (cd /home/rharish/.cache/bazel/_bazel_rharish/5d4d7b1255c710f6c814ab2f3f084405/execroot/org_tensorflow && \
  exec env - \
    LD_LIBRARY_PATH=:/usr/local/lib:/opt/cuda/lib64 \
    PATH=/home/rharish/bin:/usr/local/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/sbin:/opt/cuda/bin:/usr/lib/jvm/default/bin:/usr/bin/site_perl:/usr/bin/vendor_perl:/usr/bin/core_perl \
    PWD=/proc/self/cwd \
  external/local_config_cuda/crosstool/clang/bin/crosstool_wrapper_driver_is_not_gcc -MD -MF bazel-out/host/bin/external/protobuf_archive/_objs/python/google/protobuf/pyext/_message.so/descriptor_containers.pic.d '-frandom-seed=bazel-out/host/bin/external/protobuf_archive/_objs/python/google/protobuf/pyext/_message.so/descriptor_containers.pic.o' -iquote external/protobuf_archive -iquote bazel-out/host/genfiles/external/protobuf_archive -iquote bazel-out/host/bin/external/protobuf_archive -iquote external/bazel_tools -iquote bazel-out/host/genfiles/external/bazel_tools -iquote bazel-out/host/bin/external/bazel_tools -iquote external/local_config_python -iquote bazel-out/host/genfiles/external/local_config_python -iquote bazel-out/host/bin/external/local_config_python -isystem external/protobuf_archive/python -isystem bazel-out/host/genfiles/external/protobuf_archive/python -isystem bazel-out/host/bin/external/protobuf_archive/python -isystem external/protobuf_archive/src -isystem bazel-out/host/genfiles/external/protobuf_archive/src -isystem bazel-out/host/bin/external/protobuf_archive/src -isystem external/local_config_python/python_include -isystem bazel-out/host/genfiles/external/local_config_python/python_include -isystem bazel-out/host/bin/external/local_config_python/python_include '-std=c++11' -Wno-builtin-macro-redefined '-D__DATE__="redacted"' '-D__TIMESTAMP__="redacted"' '-D__TIME__="redacted"' -fPIC -U_FORTIFY_SOURCE '-D_FORTIFY_SOURCE=1' -fstack-protector -Wall -fno-omit-frame-pointer -no-canonical-prefixes -DNDEBUG -g0 -O2 -ffunction-sections -fdata-sections -g0 '-march=native' -g0 -DHAVE_PTHREAD -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -Wno-unused-function -Wno-writable-strings '-DGOOGLE_PROTOBUF_HAS_ONEOF=1' '-DPROTOBUF_PYTHON_ALLOW_OVERSIZE_PROTOS=1' -c external/protobuf_archive/python/google/protobuf/pyext/descriptor_containers.cc -o bazel-out/host/bin/external/protobuf_archive/_objs/python/google/protobuf/pyext/_message.so/descriptor_containers.pic.o)
external/protobuf_archive/python/google/protobuf/pyext/descriptor_containers.cc: In function 'bool google::protobuf::python::descriptor::_GetItemByKey(google::protobuf::python::PyContainer*, PyObject*, const void**)':
external/protobuf_archive/python/google/protobuf/pyext/descriptor_containers.cc:69:45: error: invalid conversion from 'const char*' to 'char*' [-fpermissive]
        ((*(charpp) = PyUnicode_AsUTF8AndSize(ob, (sizep))) == NULL? -1: 0): \
                      ~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~
external/protobuf_archive/python/google/protobuf/pyext/descriptor_containers.cc:172:13: note: in expansion of macro 'PyString_AsStringAndSize'
         if (PyString_AsStringAndSize(key, &name, &name_size) < 0) {
             ^~~~~~~~~~~~~~~~~~~~~~~~
external/protobuf_archive/python/google/protobuf/pyext/descriptor_containers.cc:69:45: error: invalid conversion from 'const char*' to 'char*' [-fpermissive]
        ((*(charpp) = PyUnicode_AsUTF8AndSize(ob, (sizep))) == NULL? -1: 0): \
                      ~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~
external/protobuf_archive/python/google/protobuf/pyext/descriptor_containers.cc:189:13: note: in expansion of macro 'PyString_AsStringAndSize'
         if (PyString_AsStringAndSize(key, &camelcase_name, &name_size) < 0) {
             ^~~~~~~~~~~~~~~~~~~~~~~~
At global scope:
cc1plus: warning: unrecognized command line option '-Wno-writable-strings'
Target //tensorflow/tools/pip_package:build_pip_package failed to build
INFO: Elapsed time: 63.634s, Critical Path: 9.08s
INFO: 464 processes: 464 local.
FAILED: Build did NOT complete successfully

My system configuration is:
OS Platform and Distribution: Arch Linux
Python: Python 3.7.0
TensorFlow installed from: source (https://github.com/tensorflow/tensorflow.git), master branch
TensorFlow version: TensorFlow 1.9.0
Bazel version: 0.16.0
CUDA/cuDNN version: CUDA 9.2
GPU model and memory: NVIDIA GeForce GTX 960M, 4GB

@PlatinumLyfe

This comment has been minimized.

Copy link

PlatinumLyfe commented Jan 8, 2019

Got python3.7.2 very recent compiled for windows with CUDA 10, links at this repo:
https://github.com/PlatinumLyfe/tf-windows-gpu/

@BhuvaneshBhatt

This comment has been minimized.

Copy link

BhuvaneshBhatt commented Jan 9, 2019

Hi, any progress on this compatibility issue?

bbhattmaclap:~ bbhatt$ pip3 install --upgrade tensorflow
Collecting tensorflow
Could not find a version that satisfies the requirement tensorflow (from versions: )
No matching distribution found for tensorflow
bbhattmaclap:~ bbhatt$

@av8ramit

This comment has been minimized.

Copy link
Member

av8ramit commented Jan 9, 2019

@BhuvaneshBhatt the latest tensorflow official package does not have Python3.7 support. You'll have to use your tf-nightly-gpu package. We are trying to have it for 1.13.

@NewUserHa

This comment has been minimized.

Copy link

NewUserHa commented Jan 10, 2019

@PlatinumLyfe unable to install.
your link only have xxx-cp36-cp36m-....
no cp37 and no -gpu- version.

@tensorflow tensorflow deleted a comment from iSevenDays Jan 11, 2019

@girving

This comment has been minimized.

Copy link
Contributor

girving commented Jan 11, 2019

Please stop asking for other people to provide compiled binaries. In addition to spamming everyone interested in when official Python 3.7 support lands, asking on a publicly commentable page for someone to send you a binary to run is not particularly secure.

@adrianodennanni

This comment has been minimized.

Copy link

adrianodennanni commented Jan 24, 2019

Tensorflow 1.13-rc0 has been released (https://github.com/tensorflow/tensorflow/releases/tag/v1.13.0-rc0), however there is no Python 3.7 build on PyPI (https://pypi.org/project/tensorflow/1.13.0rc0/#files). Will Tensorflow 1.13 be released for Python 3.7 officially?

@av8ramit

This comment has been minimized.

Copy link
Member

av8ramit commented Jan 24, 2019

We are aiming to try and have Windows and Ubuntu python binaries by rc2 or the official.

@BhuvaneshBhatt

This comment has been minimized.

Copy link

BhuvaneshBhatt commented Jan 24, 2019

@av8ramit

This comment has been minimized.

Copy link
Member

av8ramit commented Feb 6, 2019

We now have a tf-nightly CPU package for Mac. Will try and have that for rc2 as well.

@Sayan98

This comment has been minimized.

Copy link

Sayan98 commented Feb 8, 2019

To anyone else coming to this thread, the above mentioned tf-nightly packages are good. Install them with pip3 install tf-nightly for the CPU version. Tested, working on Python 3.7.2 on MacOS Mojave 10.14.2

@av8ramit

This comment has been minimized.

Copy link
Member

av8ramit commented Feb 8, 2019

The 1.13.0rc1 release includes Python3.7 binaries for all OS-es for cpu and gpu.

@MagixInTheAir

This comment has been minimized.

Copy link

MagixInTheAir commented Feb 12, 2019

I can't seem to be able to install it. Any simple troubleshooting steps ?
EDIT : Issue was the 32-bit version of CPython was used on a 64-bit system.

@av8ramit

This comment has been minimized.

Copy link
Member

av8ramit commented Feb 12, 2019

@MagixInTheAir I'm closing this issue since it's just about general Python 3.7 support. Please reopen a new issue with logs and more information about your setup if you're still having issues.

@av8ramit av8ramit closed this Feb 12, 2019

michael-k added a commit to michael-k/tensorflow that referenced this issue Feb 13, 2019

Add Python 3.7 to classifiers
Python 3.7 compatibility seems fixed in 1.13, see
eg. tensorflow#20517 and tensorflow#17022.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment