Bump TF to 2.7.0

[kvignesh1420]

This PR bumps the TensorFlow (dependency) version to 2.7.0

[yongtang]

Thanks @kvignesh1420 . The following in WORKSPACE may need to be updated as well https://github.com/tensorflow/io/blob/master/WORKSPACE#L154-L161

[yongtang]

@kvignesh1420 There is also a PR in tensorflow core that might be needed to fix the header file inclusion issue, in order to update TF dependency to 2.7.0:

tensorflow/tensorflow#52313

cc @mihaimaruseac

[kvignesh1420]

I see, thanks for the info @yongtang. Will keep this PR on hold until then.

[kvignesh1420]

@yongtang I also see that there are no tensorflow 2.7.0rc0 wheels for python3.6.

[mihaimaruseac]

TF no longer releases for py36.

[yongtang]

PR tensorflow/tensorflow#52628 is related.

[kvignesh1420]

@yongtang thanks!

[yongtang]

@kvignesh1420 One issue with the linux build can be fixed with the following diff i think:

% git diff HEAD~1
diff --git a/third_party/toolchains/tf/BUILD.tpl b/third_party/toolchains/tf/BUILD.tpl
index a4dcc25e..03935e80 100644
--- a/third_party/toolchains/tf/BUILD.tpl
+++ b/third_party/toolchains/tf/BUILD.tpl
@@ -7,6 +7,7 @@ cc_library(
     visibility = ["//visibility:public"],
     deps = [
         "@com_google_absl//absl/container:flat_hash_map",
+        "@com_google_absl//absl/container:flat_hash_set",
         "@com_google_absl//absl/container:inlined_vector",
         "@com_google_absl//absl/strings",
         "@com_google_absl//absl/strings:cord",

[yongtang]

The following will fix another build failure:

% git diff HEAD~1
diff --git a/.github/workflows/build.bazel.sh b/.github/workflows/build.bazel.sh
index 7550af91..5687b2d6 100755
--- a/.github/workflows/build.bazel.sh
+++ b/.github/workflows/build.bazel.sh
@@ -15,7 +15,14 @@
 # ==============================================================================
 set -e -x
 
-export TENSORFLOW_INSTALL="$(python3 setup.py --install-require)"
+
+PYTHON=python3
+if [[ $# == 1 ]]; then
+    PYTHON=$1
+fi
+$PYTHON --version
+
+export TENSORFLOW_INSTALL="$($PYTHON setup.py --install-require)"
 
 export BAZEL_OS=$(uname | tr '[:upper:]' '[:lower:]')
 export BAZEL_VERSION=$(cat .bazelversion)
@@ -23,15 +30,15 @@ curl -sSOL https://github.com/bazelbuild/bazel/releases/download/${BAZEL_VERSION
 bash -e bazel-${BAZEL_VERSION}-installer-${BAZEL_OS}-x86_64.sh
 bazel version
 
-python3 -m pip --version
+$PYTHON -m pip --version
 
-python3 -m pip install --upgrade pip
-python3 -m pip install --upgrade setuptools
-python3 -m pip --version
+$PYTHON -m pip install --upgrade pip
+$PYTHON -m pip install --upgrade setuptools
+$PYTHON -m pip --version
 
-python3 -m pip install -q ${TENSORFLOW_INSTALL}
+$PYTHON -m pip install -q ${TENSORFLOW_INSTALL}
 
-python3 tools/build/configure.py
+$PYTHON tools/build/configure.py
 
 cat .bazelrc
 
diff --git a/.github/workflows/build.yml b/.github/workflows/build.yml
index 548c19cc..91153a2f 100644
--- a/.github/workflows/build.yml
+++ b/.github/workflows/build.yml
@@ -208,7 +208,7 @@ jobs:
     name: Bazel Linux
     runs-on: ubuntu-latest
     container:
-      image: gcr.io/tensorflow-testing/nosla-ubuntu16.04-manylinux2010@sha256:82fc19f511e1a78eb3e1e077ac2ffc0733efd05c4d1d933ec186cb1dd00625ed
+      image: gcr.io/tensorflow-testing/nosla-ubuntu16.04-manylinux2010
       env:
         REPO_NAME: ${{ env.REPO_NAME }}
         EVENT_NAME: ${{ env.EVENT_NAME }}
@@ -227,7 +227,7 @@ jobs:
           if [[ "${EVENT_NAME}" == "push" && "${REPO_NAME}" == "tensorflow/io" ]]; then
             export BAZEL_OPTIMIZATION="$BAZEL_OPTIMIZATION --remote_upload_local_results=true --google_credentials=service_account_creds.json"
           fi
-          bash -x -e .github/workflows/build.bazel.sh
+          bash -x -e .github/workflows/build.bazel.sh python3.8
           sudo cp .bazelrc build/tensorflow_io/
       - uses: actions/upload-artifact@v2
         with:

[kvignesh1420]

@yongtang thanks! the macOS builds are passing locally but the CI builds fail due to:

ERROR: /private/var/tmp/_bazel_runner/9d136bbef824d938aef9fac8a59ffcc5/external/local_config_python/BUILD:69:8: in outs attribute of genrule rule @local_config_python//:python_include: Genrules without outputs don't make sense
ERROR: Analysis of target '//tensorflow_io/core/grpc:endpoint_py_pb2_grpc' failed; build aborted: Analysis of target '@local_config_python//:python_include' failed

did you encounter such errors before?

[kvignesh1420]

also, is there a way to check the cached bazel-bin files? Seems like the bazel-bin/external/local_config_python/python_include directory is not being created/present.

[yongtang]

@kvignesh1420 The macOS build failure seems to be caused by macOS's default python3 and the installed python3 in GitHub Actions. I think the following diff will resolve the macOS build failure:

% git diff HEAD~1
diff --git a/.github/workflows/build.yml b/.github/workflows/build.yml
index bc868cce..32cad026 100644
--- a/.github/workflows/build.yml
+++ b/.github/workflows/build.yml
@@ -115,8 +115,6 @@ jobs:
           if [[ "${EVENT_NAME}" == "push" && "${REPO_NAME}" == "tensorflow/io" ]]; then
             export BAZEL_OPTIMIZATION="$BAZEL_OPTIMIZATION --remote_upload_local_results=true --google_credentials=service_account_creds.json"
           fi
-          echo "Bring /usr/bin to front as GitHub does not use system python3 by default"
-          export PATH=/usr/bin:$PATH
           sudo -E -H bash -e .github/workflows/build.bazel.sh
       - uses: actions/upload-artifact@v2
         with:

[yongtang]

@kvignesh1420 For windows build failures, it looks like the cache of the bazel is not reliable on Windows. With the following diff the Windows build will work:

git diff HEAD~1
diff --git a/.github/workflows/build.yml b/.github/workflows/build.yml
index 32cad026..7f1281c0 100644
--- a/.github/workflows/build.yml
+++ b/.github/workflows/build.yml
@@ -324,10 +324,10 @@ jobs:
         shell: cmd
         run: |
           @echo on
-          set "BAZEL_OPTIMIZATION=--config=cache"
+          set "BAZEL_OPTIMIZATION="
           if "%EVENT_NAME%" == "push" (
             if "%REPO_NAME%" == "tensorflow/io" (
-              set "BAZEL_OPTIMIZATION=%BAZEL_OPTIMIZATION% --remote_upload_local_results=true --google_credentials=service_account_creds.json"
+              set "BAZEL_OPTIMIZATION=--remote_upload_local_results=true --google_credentials=service_account_creds.json"
             )
           )
           set /P BAZEL_VERSION=< .bazelversion

[yongtang]

@kvignesh1420 With the remaining build failures, it looks like removing tests/__init__.py will resolve the issue. Not exactly sure why this is the case though can you give it a try?

[yongtang]

Able to see build pass by removing tests/__init__.py file:

https://github.com/yongtang/io/actions/runs/1437952005

[kvignesh1420]

@yongtang thanks! although it's surprising that the default behaviours have changed so much w.r.t the GitHub virtual environments. Related discussion : actions/runner-images#4060