Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Kernel Restart - Incompatibility between nx.draw and utils_tf.data_dicts_to_graphs_tuple #124

Closed
mshearer0 opened this issue Aug 3, 2020 · 7 comments

Comments

@mshearer0
Copy link

Hi.

I'm trying to use nx.draw and utils_tf.data_dicts_to_graphs_tuple in the same TF2 notebook.

Whichever is executed second seems to cause a kernel restart in the notebook which i can't explain. Importing networkx is fine as long as nx.draw is not run.

@Mistobaan - I get this behaviour on your very helpful TF2 version of graph_nets_basic tutorial.

Michael.

@alvarosg
Copy link
Collaborator

I have not observed this, not sure if @Mistobaan did.

Are you running on your own kernel, or on Google Colaboratory?

@Mistobaan
Copy link
Contributor

In my experience that is usually an out of memory case. Check the system logs if you are running on Colab.

@mshearer0
Copy link
Author

Hi, thanks. I'm running on GCP Notebook with 15GB RAM. GCP logs show:

Aug 12 21:12:02 ... bash[1278]: OMP: Error #15: Initializing libiomp5.so, but found libomp.so already initialized.
Aug 12 21:12:02 ... bash[1278]: OMP: Hint This means that multiple copies of the OpenMP runtime have been linked into the progr
am. That is dangerous, since it can degrade performance or cause incorrect results. The best thing to do is to ensure that only a single OpenMP runtime is
linked into the process, e.g. by avoiding static linking of the OpenMP runtime in any library. As an unsafe, unsupported, undocumented workaround you can
set the environment variable KMP_DUPLICATE_LIB_OK=TRUE to allow the program to continue to execute, but that may cause crashes or silently produce incorr
ect results. For more information, please see http://www.intel.com/software/products/support/.
Aug 12 21:12:03 ... bash[1278]: [I 21:12:03.530 LabApp] KernelRestarter: restarting kernel (1/5), keep random ports
Aug 12 21:12:03 ... bash[1278]: kernel ... restarted

@Mistobaan
Copy link
Contributor

I think the answer is printed by your logs:

set the environment variable KMP_DUPLICATE_LIB_OK=TRUE to allow the program to continue to execute, but that may cause crashes or silently produce incorrect results

@mshearer0
Copy link
Author

@Mistobaan - yes, I’ve used that as a workaround but wondered if there was a better option?

@Mistobaan
Copy link
Contributor

Get a bigger machine with more memory? Can you replicate the problem into a colab and post the link to the colab? make sure you set the share permissions.

@mshearer0
Copy link
Author

Upgrading to GCP Notebook Tensorflow 2.3 (from 2.2.0) resolved the issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants