Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[TF 2.0.0-rc0] Cannot find any 2.0.0 RC0 API references. #31973

Closed
makercob opened this issue Aug 26, 2019 · 46 comments · Fixed by #34629
Closed

[TF 2.0.0-rc0] Cannot find any 2.0.0 RC0 API references. #31973

makercob opened this issue Aug 26, 2019 · 46 comments · Fixed by #34629
Assignees

Comments

@makercob
Copy link

@makercob makercob commented Aug 26, 2019

System information

  • Have I written custom code (as opposed to using a stock example script provided in TensorFlow): Yes.
  • OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Windows 10 and macOS
  • TensorFlow installed from (source or binary): pip
  • TensorFlow version (use command below): 2.0.0-rc0 (both GPU and CPU versions)
  • Python version: 3.6.8

Describe the current behavior
Not a single reference to members of tensorflow can be found in IDE(PyCharm).
However, programs run as expected.

Describe the expected behavior
References to members of tensorflow can be found.

Screen Shot 2019-08-26 at 17 13 30

@calmisential

This comment has been minimized.

Copy link

@calmisential calmisential commented Aug 26, 2019

@makercob Hello, friend! I have the same problem. I found PyCharm cannot find the tensorflow_2.0.0-rc0 API, but jupyter notebook and jupyter lab can. I'm also puzzled by this problem.

@georgealexandruvlad

This comment has been minimized.

Copy link

@georgealexandruvlad georgealexandruvlad commented Aug 26, 2019

Many others (including myself) reported the same problem in the following thread #26502. The previous issue is closed so I will leave this here hoping it has better coverage being an open issue.

@makercob

This comment has been minimized.

Copy link
Author

@makercob makercob commented Aug 26, 2019

@georgealexandruvlad uhmmm... PyCharm to blame? but 2.0.0b1 worked fine.

@georgealexandruvlad

This comment has been minimized.

Copy link

@georgealexandruvlad georgealexandruvlad commented Aug 26, 2019

@makercob well, you're right. I'm just looking for a solution for large projects which can't be implemented in jupyter/google colab. As I stated in my comment in the other thread:

If it is any workaround on this issue until a fix is released (from tensorflow team or PyCharm) I would be grateful to use it.

I only ask for a solution to get it working because there are lots of good new features in the new release. This could mean a solution from the tensorflow team if there are some steps to be followed in order to make it work with PyCharm or another IDE in which they know it is working properly.

@Efaq

This comment has been minimized.

Copy link

@Efaq Efaq commented Aug 26, 2019

Is there any workaround for that, while it is not solved?

EDIT: I simply went back to beta for the moment, the references work without problems there.

@makercob

This comment has been minimized.

Copy link
Author

@makercob makercob commented Aug 26, 2019

@georgealexandruvlad Could you please share a link to release notes of 2.0.0-rc0? I just can't find it anywhere. Thanks!

@georgealexandruvlad

This comment has been minimized.

Copy link

@georgealexandruvlad georgealexandruvlad commented Aug 26, 2019

@makercob Sure. I think this should be what you're searching for: https://github.com/tensorflow/tensorflow/releases

@makercob

This comment has been minimized.

Copy link
Author

@makercob makercob commented Aug 26, 2019

@Efaq Any alternative to PyCharm? VS Code works not so well for me.

@mihaimaruseac mihaimaruseac self-assigned this Aug 26, 2019
@mihaimaruseac

This comment has been minimized.

Copy link
Contributor

@mihaimaruseac mihaimaruseac commented Aug 26, 2019

We understand that this is a serious issue and we are actively trying to solve this. However, this is not something we can get in by the final release of 2.0 as the issue is an unfortunate side effect of several components: having to support both v1 and v2 code until the final release of 2.0, having to add deprecation wrapping to a lot of API endpoints in v1, adding the virtual pip package to help with modularization efforts and adding lazy loading to speed up import tensorflow as tf time.

That being said, once 2.0 is released we can clean up a large part of the code and can then solve this more efficient. Furthermore, once TensorFlow gets more modular we can clean up other parts of the code to make solving this issue more tractable. Unfortunately, we don't have a clear timeline for the second part, so we don't know exactly when we can say with 100% certainty that this issue gets solved.

However, @annarev has submitted a change (1171258) that will enable autocomplete in a majority of use cases but we ran into some issues and it has been reverted (1d1f7df). We will try to get this change submitted again in the coming days and then cherry-pick it on the next release candidate.

In any case, as we get new releases on the TF2.* series, this issue will get solved. Sorry for the delay and for the issues it causes

@Efaq

This comment has been minimized.

Copy link

@Efaq Efaq commented Aug 27, 2019

@makercob I will try not to change, I have some preference for PyCharm. But in case it gets too annoying not to have the last version of TF, I will choose between VS and Spyder. BUT I don't even know if this would solve the issue :)

What issues have you had with VS Code? Could you get autocompletion?

@makercob

This comment has been minimized.

Copy link
Author

@makercob makercob commented Aug 27, 2019

@Efaq No autocompletion with VS Code either. :-/
Spoiled by PyCharm, not everything seems right to me in VS Code when working with Python. say, instant variable evaluation and numpy array inspection.

@shianiawhite

This comment has been minimized.

Copy link

@shianiawhite shianiawhite commented Aug 30, 2019

@makercob Hello, friend! I have the same problem. I found PyCharm cannot find the tensorflow_2.0.0-rc0 API, but jupyter notebook and jupyter lab can. I'm also puzzled by this problem.

Just a quick note about the general reason of why this happening. Jupyter dynamically figures out all the packages. PyCharm (for speed reasons) statically indexes everything. If things are only defined when dynamically run, PyCharm won't see them. Python packages are generally encouraged to have these things available statically. This explanation is slightly inaccurate, but close enough.

Unfortunately, in my case, I value PyCharm's capabilities and workflow far more than any improvements and fixes in the new TensorFlow releases. So this is a blocker on me upgrading.

@shianiawhite

This comment has been minimized.

Copy link

@shianiawhite shianiawhite commented Aug 30, 2019

Also, as noted by others, this will not be a PyCharm specific issue. Many IDEs expect to be able to statically resolve the packages.

@deepgradient

This comment has been minimized.

Copy link

@deepgradient deepgradient commented Sep 1, 2019

Also, as noted by others, this will not be a PyCharm specific issue. Many IDEs expect to be able to statically resolve the packages.

Exactly, VScode has the same issue! :( why google use such goofy importing method in tensorflow?

@mihaimaruseac

This comment has been minimized.

Copy link
Contributor

@mihaimaruseac mihaimaruseac commented Sep 1, 2019

We needed a way to separate private internal APIs with no guarantees on stability with public stable(r) APIs so that code that works on Tensorflow x.y also works on Tensorflow x.{y+1} while evolving in private.

We also needed a way to suppot both a preview of 2.0 and 1.* at the same time.

But most of these needs will go away with the release of 2.0. Also, there is ongoing work to modularize TensorFlow and one of the components of that work is extracting the public API to a separate module. After that, autocomplete should work perfectly. See tensorflow/community#77

@deepgradient

This comment has been minimized.

Copy link

@deepgradient deepgradient commented Sep 1, 2019

We needed a way to separate private internal APIs with no guarantees on stability with public stable(r) APIs so that code that works on Tensorflow x.y also works on Tensorflow x.{y+1} while evolving in private.

We also needed a way to suppot both a preview of 2.0 and 1.* at the same time.

But most of these needs will go away with the release of 2.0. Also, there is ongoing work to modularize TensorFlow and one of the components of that work is extracting the public API to a separate module. After that, autocomplete should work perfectly. See tensorflow/community#77

Thanks for your clarification. I hope we can have everything perfect soon.
Do you have any idea as how long it may take to completely resolve this issue?

@mihaimaruseac

This comment has been minimized.

Copy link
Contributor

@mihaimaruseac mihaimaruseac commented Sep 2, 2019

Probably end of year

@annarev

This comment has been minimized.

Copy link
Member

@annarev annarev commented Sep 3, 2019

The fix for a large part of autocomplete has been submitted here: 386da97 and should be in latest nightly. However, there are still some parts that don't work and we are looking into them.

@calmisential

This comment has been minimized.

Copy link

@calmisential calmisential commented Sep 5, 2019

The fix for a large part of autocomplete has been submitted here: 386da97 and should be in latest nightly. However, there are still some parts that don't work and we are looking into them.

Thanks for your work! Does this mean that the autocomplete problem will be solved when the next tensorflow-2.0 version (may be rc1) is released ?

@makercob

This comment has been minimized.

Copy link
Author

@makercob makercob commented Sep 6, 2019

@calmisential Nope. Spring is indeed pretty much flexible.

@egeozsoy

This comment has been minimized.

Copy link

@egeozsoy egeozsoy commented Sep 7, 2019

A workaround which seems to work for now is importing as directly as possible. This means instead of
from tensorflow.keras import Model
one has to do
from tensorflow_core.python.keras.models import Model

Instead of
import tensorflow as tf and then
tf.keras.optimizers.Adam()

Use
from tensorflow_core.python.keras.optimizers import Adam
and then just
Adam()

It makes it more cumbersome to import stuff, but at least autocomplete and other IDE features work

@RunOrVeith

This comment has been minimized.

Copy link

@RunOrVeith RunOrVeith commented Sep 13, 2019

Is this fixed for you in rc1? I still have a similar problem.
from tensorflow import keras works
from tensorflow.keras import layers does not work.

@NiclasLaurenzChrist

This comment has been minimized.

Copy link

@NiclasLaurenzChrist NiclasLaurenzChrist commented Sep 13, 2019

@RunOrVeith as mentioned the problem remains in rc1 (and will probably until the release of 2.0)

@RunOrVeith

This comment has been minimized.

Copy link

@RunOrVeith RunOrVeith commented Sep 13, 2019

Ok I thought that might have changed because the release notes state
Fixes autocomplete for most TensorFlow API references by switching to use relative imports in API __init__.py files.

@chuxi

This comment has been minimized.

Copy link

@chuxi chuxi commented Sep 13, 2019

Chinese to such interesting problem: He He. 呵呵

@jubjamie

This comment has been minimized.

Copy link
Contributor

@jubjamie jubjamie commented Sep 17, 2019

The fix for a large part of autocomplete has been submitted here: 386da97 and should be in latest nightly. However, there are still some parts that don't work and we are looking into them.

Just to confirm, this is no longer working in the nightly release is it?

@annarev

This comment has been minimized.

Copy link
Member

@annarev annarev commented Sep 17, 2019

Autocomplete has been fixed for import tensorflow as tf and from tensorflow import keras in rc1.
However, from tensorflow.keras import layers won't be fixed even in 2.0 final. It requires major changes to TensorFlow (renaming lite/ directory) to get it working. These changes are too big for this release since it only accepts critical cherrypicks.

Also, recommended way to import TensorFlow is import tensorflow as tf.

@jubjamie

This comment has been minimized.

Copy link
Contributor

@jubjamie jubjamie commented Sep 18, 2019

Just updated to rc1 and still have no success with from tensorflow import keras or import tensorflow as tf. This is gpu version if it makes a difference.
I've rebuilt all indices and caches etc. to eliminate a PyCharm issue. Changing interpreter back to 1.14 goes straight back to the intended functionality.

@georgesterpu

This comment has been minimized.

Copy link
Contributor

@georgesterpu georgesterpu commented Sep 18, 2019

Hi, I've been coding in TF 2.0 for the past few days and used this cheap workaround for my (momentarily) limited use case: install tensorflow-gpu==2.0.0-beta1 alongside with tensorflow==2.0.0-rc1. But it's quite strange, better wait for a proper fix.

@RedTachyon

This comment has been minimized.

Copy link

@RedTachyon RedTachyon commented Sep 23, 2019

This has been fixed for rc1 in the newest PyCharm EAP: https://blog.jetbrains.com/pycharm/2019/09/2019-3-eap-2

@Xovee

This comment has been minimized.

Copy link

@Xovee Xovee commented Sep 23, 2019

This has been fixed for rc1 in the newest PyCharm EAP: https://blog.jetbrains.com/pycharm/2019/09/2019-3-eap-2

It works, thx a lot.

@mihaimaruseac

This comment has been minimized.

Copy link
Contributor

@mihaimaruseac mihaimaruseac commented Sep 23, 2019

Closing as it has been fixed. Thanks for verifying.

@shianiawhite

This comment has been minimized.

Copy link

@shianiawhite shianiawhite commented Sep 23, 2019

Just to note, although PyCharm now correctly resolves TensorFlow, this solution does not solve the issue for other editors. There are others in this thread who have mentioned this issue also affects VS Code (and likely other editors).

(... but I use PyCharm, so I'm satisfied)

@mihaimaruseac

This comment has been minimized.

Copy link
Contributor

@mihaimaruseac mihaimaruseac commented Sep 23, 2019

In that case, reopening.

@speediedan

This comment has been minimized.

Copy link

@speediedan speediedan commented Sep 24, 2019

While the workaround of importing tensorflow_core to access API functionality normally under tf.config (e.g.

from tensorflow_core.python.framework import config as tfconfig

allows autocompletion to function (tf v2.0.0-rc2, pycharm EAP: 193.3519.27), it does not allow me to use tf.config functionality due to the import error below:

ImportError: cannot import name 'module_util' from 'tensorflow.python.tools' (/opt/anaconda/envs/tf2rc/lib/python3.7/site-packages/tensorflow/python/tools/__init__.py)

Any suggestions on a temporary workaround for accessing tf.config functionality? Thanks very much in advance!!

@mihaimaruseac

This comment has been minimized.

Copy link
Contributor

@mihaimaruseac mihaimaruseac commented Sep 24, 2019

Importing directly from tensorflow_core is not supported and will result in your code being broken by future upgrades.

@speediedan

This comment has been minimized.

Copy link

@speediedan speediedan commented Sep 25, 2019

Thanks Mihai, understood, I'm just looking for a temporary workaround to access tf.config, which isn't working with rc2. I think this is actually a separate issue and I'll create a separate ticket accordingly. Apologies for the confusion.

@ElPapi42

This comment has been minimized.

Copy link

@ElPapi42 ElPapi42 commented Oct 1, 2019

This problem persist with Tensorflow 2.0 lastest release, using VS Code

@makercob

This comment has been minimized.

Copy link
Author

@makercob makercob commented Oct 2, 2019

@ElPapi42 PyCharm 2019.3 coming to rescue, probably in Nov.

@juhai

This comment has been minimized.

Copy link

@juhai juhai commented Oct 17, 2019

I can reproduce this problem with tensorflow==1.15.0, using latest official PyCharm release (2019-2.3). Is the fix going to be applied to 1.X branch?

@soldier828

This comment has been minimized.

Copy link

@soldier828 soldier828 commented Oct 28, 2019

This problem persist with Tensorflow 2.0 lastest release, using VS Code
Yes, too mad with this issue

@arbellea

This comment has been minimized.

Copy link

@arbellea arbellea commented Nov 11, 2019

Any timeline of a fix for this issue?

@episodeyang

This comment has been minimized.

Copy link

@episodeyang episodeyang commented Nov 17, 2019

The fix is to use import tensorflow.compat.v2 as tf instead of the regular import.

@ElPapi42

This comment has been minimized.

Copy link

@ElPapi42 ElPapi42 commented Nov 20, 2019

The fix is to use import tensorflow.compat.v2 as tf instead of the regular import.

Not working for me :/

@davidhalter

This comment has been minimized.

Copy link

@davidhalter davidhalter commented Nov 26, 2019

We're having discussions about tensorflow2 completions in Jedi. I proposed two solutions in davidhalter/jedi#1391 (comment):

One way to fix this would be to just replace keras = _sys.modules["tensorflow.keras"] with from tensorflow import keras. This does essentially the same thing.

It's also nice to use something like:

import typing

if not typing.TYPE_CHECKING:
    keras = _sys.modules["tensorflow.keras"]

This only works in Python 3 (typing needs to be installed).

@lgeiger

This comment has been minimized.

Copy link
Contributor

@lgeiger lgeiger commented Nov 27, 2019

@davidhalter Thanks for helping debug this.

@mihaimaruseac I think I found a solution that get's rid of the problematic workarounds in #34629

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
You can’t perform that action at this time.