Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

word2vec / universal sentence encoder example? #18

Open
Jonathhhan opened this issue Jun 11, 2022 · 8 comments
Open

word2vec / universal sentence encoder example? #18

Jonathhhan opened this issue Jun 11, 2022 · 8 comments

Comments

@Jonathhhan
Copy link
Contributor

Jonathhhan commented Jun 11, 2022 •
edited

I wonder if somebody use a word2vec model with ofxTensorflow2?
I found a model and tried to use it, but without success: https://tfhub.dev/google/Wiki-words-250-with-normalization/2
For now I use ofxWord2vec https://github.com/perevalovds/ofxWord2Vec for finding subtitle similarities (with a pretrained model) and I would like to do that with ofxTensorflow2 too...
https://github.com/Jonathhhan/ofEmscriptenExamples/tree/main/montageautomat_threaded_2
@bytosaur
Copy link
Member

Maybe I can help to fix it. At which stage are you stuck? Please add error messages if that's the case.

@Jonathhhan
Copy link
Contributor Author

Jonathhhan commented Jun 12, 2022 •
edited

@bytosaur thanks. Actually I am stuck at an very early stage, it crashes when I try to run the model (I expected to get the tensor of the string):

	if (!model2.load("model2")) {
		std::exit(EXIT_FAILURE);
	}

	auto s1 = cppflow::tensor(std::string("hello"));
	auto s2 = cppflow::tensor(std::string("tree"));
	auto s3 = cppflow::tensor(std::string("house"));
	auto s = cppflow::concat(0, { s1, s2, s3 });
	std::cout << cppflow::concat(0, { s1, s2, s3 });

	auto output2 = model2.runModel(s);

I have to admit that my approach is very naive, and maybe you can point to a tutorial for understanding tensorflow/word2vec basics better?
What I want to achieve: Get the vector of one or several words that are part of the trained model and compare it with another word or sentence for similarity. Or, in other words: Compare a lot of subtitles and find the most similar one (which works well with my ofxWord2Vec example - here is an online version: https://montageautomat3.handmadeproductions.de/).

@bytosaur
Copy link
Member

hey @Jonathhhan,

could you please provide the error message as well? or is it just a seg fault? also which line throws the error?

I have never worked with word embeddings before, but i think your understanding is correct. The text gets transformed into an n-dimensional vector which can be used for comparison. Usually these embeddings are used as a first step in language models to have a latent representation of the text.

@Jonathhhan
Copy link
Contributor Author

@bytosaur it crashes at auto output2 = model2.runModel(s); with Debug Error! - abort() has been called
same with model2.printOperations(); but it prints the operations, which are far less than in the example model:

[notice ] ofxTensorFlow2: ====== Model Operations with Shapes ======
[notice ] ofxTensorFlow2: hash_table with shape: []
[notice ] ofxTensorFlow2: Variable with shape: []
[notice ] ofxTensorFlow2: Variable/Read/ReadVariableOp with shape: [1009375, 250]
[notice ] ofxTensorFlow2: asset_path_initializer with shape: []
[notice ] ofxTensorFlow2: Variable_1 with shape: []
[notice ] ofxTensorFlow2: Variable_1/IsInitialized/VarIsInitializedOp with shape: []

@Jonathhhan
Copy link
Contributor Author

Jonathhhan commented Jun 14, 2022 •
edited

If I load the universal sentence encoder model https://tfhub.dev/google/universal-sentence-encoder/4 instead, I get a lot of information with model.printOperations(); and it does not crash (but if I call runModel(input) it crashes, too):

[notice ] ofxTensorFlow2: ====== Model Operations with Shapes ======
[notice ] ofxTensorFlow2: unused_resource with shape: []
[notice ] ofxTensorFlow2: unused_resource_1 with shape: []
[notice ] ofxTensorFlow2: save_counter with shape: []
[notice ] ofxTensorFlow2: save_counter/Read/ReadVariableOp with shape: []
[notice ] ofxTensorFlow2: Embeddings/sharded_0 with shape: []
[notice ] ofxTensorFlow2: Embeddings/sharded_0/Read/ReadVariableOp with shape: [26667, 320]
[notice ] ofxTensorFlow2: Embeddings/sharded_1 with shape: []
[notice ] ofxTensorFlow2: Embeddings/sharded_1/Read/ReadVariableOp with shape: [26667, 320]
[notice ] ofxTensorFlow2: Embeddings/sharded_2 with shape: []
[notice ] ofxTensorFlow2: Embeddings/sharded_2/Read/ReadVariableOp with shape: [26667, 320]
[notice ] ofxTensorFlow2: Embeddings/sharded_3 with shape: []
[notice ] ofxTensorFlow2: Embeddings/sharded_3/Read/ReadVariableOp with shape: [26667, 320]
[notice ] ofxTensorFlow2: Embeddings/sharded_4 with shape: []
[notice ] ofxTensorFlow2: Embeddings/sharded_4/Read/ReadVariableOp with shape: [26667, 320]
[notice ] ofxTensorFlow2: Embeddings/sharded_5 with shape: []
[notice ] ofxTensorFlow2: Embeddings/sharded_5/Read/ReadVariableOp with shape: [26667, 320]
[notice ] ofxTensorFlow2: Embeddings/sharded_6 with shape: []
[notice ] ofxTensorFlow2: Embeddings/sharded_6/Read/ReadVariableOp with shape: [26667, 320]
[notice ] ofxTensorFlow2: Embeddings/sharded_7 with shape: []
[notice ] ofxTensorFlow2: Embeddings/sharded_7/Read/ReadVariableOp with shape: [26667, 320]
[notice ] ofxTensorFlow2: Embeddings/sharded_8 with shape: []
[notice ] ofxTensorFlow2: Embeddings/sharded_8/Read/ReadVariableOp with shape: [26667, 320]
[notice ] ofxTensorFlow2: Embeddings/sharded_9 with shape: []
[notice ] ofxTensorFlow2: Embeddings/sharded_9/Read/ReadVariableOp with shape: [26667, 320]
[notice ] ofxTensorFlow2: Embeddings/sharded_10 with shape: []
[notice ] ofxTensorFlow2: Embeddings/sharded_10/Read/ReadVariableOp with shape: [26667, 320]
[notice ] ofxTensorFlow2: Embeddings/sharded_11 with shape: []
[notice ] ofxTensorFlow2: Embeddings/sharded_11/Read/ReadVariableOp with shape: [26667, 320]
[notice ] ofxTensorFlow2: Embeddings/sharded_12 with shape: []
[notice ] ofxTensorFlow2: Embeddings/sharded_12/Read/ReadVariableOp with shape: [26667, 320]
[notice ] ofxTensorFlow2: Embeddings/sharded_13 with shape: []
[notice ] ofxTensorFlow2: Embeddings/sharded_13/Read/ReadVariableOp with shape: [26667, 320]
[notice ] ofxTensorFlow2: Embeddings/sharded_14 with shape: []
[notice ] ofxTensorFlow2: Embeddings/sharded_14/Read/ReadVariableOp with shape: [26667, 320]
[notice ] ofxTensorFlow2: Embeddings/sharded_15 with shape: []
[notice ] ofxTensorFlow2: Embeddings/sharded_15/Read/ReadVariableOp with shape: [26667, 320]
[notice ] ofxTensorFlow2: Embeddings/sharded_16 with shape: []
[notice ] ofxTensorFlow2: Embeddings/sharded_16/Read/ReadVariableOp with shape: [26667, 320]
[notice ] ofxTensorFlow2: Embeddings/sharded_17 with shape: []
[notice ] ofxTensorFlow2: Embeddings/sharded_17/Read/ReadVariableOp with shape: [26667, 320]
[notice ] ofxTensorFlow2: Embeddings/sharded_18 with shape: []
[notice ] ofxTensorFlow2: Embeddings/sharded_18/Read/ReadVariableOp with shape: [26667, 320]
[notice ] ofxTensorFlow2: Embeddings/sharded_19 with shape: []
[notice ] ofxTensorFlow2: Embeddings/sharded_19/Read/ReadVariableOp with shape: [26667, 320]
[notice ] ofxTensorFlow2: Embeddings/sharded_20 with shape: []
[notice ] ofxTensorFlow2: Embeddings/sharded_20/Read/ReadVariableOp with shape: [26667, 320]
[notice ] ofxTensorFlow2: Embeddings/sharded_21 with shape: []
[notice ] ofxTensorFlow2: Embeddings/sharded_21/Read/ReadVariableOp with shape: [26667, 320]
[notice ] ofxTensorFlow2: Embeddings/sharded_22 with shape: []
[notice ] ofxTensorFlow2: Embeddings/sharded_22/Read/ReadVariableOp with shape: [26667, 320]
[notice ] ofxTensorFlow2: Embeddings/sharded_23 with shape: []
[notice ] ofxTensorFlow2: Embeddings/sharded_23/Read/ReadVariableOp with shape: [26667, 320]
[notice ] ofxTensorFlow2: Embeddings/sharded_24 with shape: []
[notice ] ofxTensorFlow2: Embeddings/sharded_24/Read/ReadVariableOp with shape: [26667, 320]
[notice ] ofxTensorFlow2: Embeddings/sharded_25 with shape: []
[notice ] ofxTensorFlow2: Embeddings/sharded_25/Read/ReadVariableOp with shape: [26667, 320]
[notice ] ofxTensorFlow2: Embeddings/sharded_26 with shape: []
[notice ] ofxTensorFlow2: Embeddings/sharded_26/Read/ReadVariableOp with shape: [26667, 320]
[notice ] ofxTensorFlow2: Embeddings/sharded_27 with shape: []
[notice ] ofxTensorFlow2: Embeddings/sharded_27/Read/ReadVariableOp with shape: [26667, 320]
[notice ] ofxTensorFlow2: Embeddings/sharded_28 with shape: []
[notice ] ofxTensorFlow2: Embeddings/sharded_28/Read/ReadVariableOp with shape: [26667, 320]
[notice ] ofxTensorFlow2: Embeddings/sharded_29 with shape: []
[notice ] ofxTensorFlow2: Embeddings/sharded_29/Read/ReadVariableOp with shape: [26667, 320]
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_0/dense/kernel/part_0 with shape: []
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_0/dense/kernel/part_0/Read/ReadVariableOp with shape: [11, 320]
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_0/dense/kernel/part_1 with shape: []
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_0/dense/kernel/part_1/Read/ReadVariableOp with shape: [11, 320]
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_0/dense/kernel/part_2 with shape: []
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_0/dense/kernel/part_2/Read/ReadVariableOp with shape: [11, 320]
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_0/dense/kernel/part_3 with shape: []
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_0/dense/kernel/part_3/Read/ReadVariableOp with shape: [11, 320]
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_0/dense/kernel/part_4 with shape: []
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_0/dense/kernel/part_4/Read/ReadVariableOp with shape: [11, 320]
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_0/dense/kernel/part_5 with shape: []
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_0/dense/kernel/part_5/Read/ReadVariableOp with shape: [11, 320]
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_0/dense/kernel/part_6 with shape: []
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_0/dense/kernel/part_6/Read/ReadVariableOp with shape: [11, 320]
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_0/dense/kernel/part_7 with shape: []
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_0/dense/kernel/part_7/Read/ReadVariableOp with shape: [11, 320]
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_0/dense/kernel/part_8 with shape: []
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_0/dense/kernel/part_8/Read/ReadVariableOp with shape: [11, 320]
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_0/dense/kernel/part_9 with shape: []
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_0/dense/kernel/part_9/Read/ReadVariableOp with shape: [11, 320]
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_0/dense/kernel/part_10 with shape: []
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_0/dense/kernel/part_10/Read/ReadVariableOp with shape: [11, 320]
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_0/dense/kernel/part_11 with shape: []
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_0/dense/kernel/part_11/Read/ReadVariableOp with shape: [11, 320]
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_0/dense/kernel/part_12 with shape: []
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_0/dense/kernel/part_12/Read/ReadVariableOp with shape: [11, 320]
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_0/dense/kernel/part_13 with shape: []
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_0/dense/kernel/part_13/Read/ReadVariableOp with shape: [11, 320]
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_0/dense/kernel/part_14 with shape: []
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_0/dense/kernel/part_14/Read/ReadVariableOp with shape: [11, 320]
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_0/dense/kernel/part_15 with shape: []
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_0/dense/kernel/part_15/Read/ReadVariableOp with shape: [11, 320]
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_0/dense/kernel/part_16 with shape: []
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_0/dense/kernel/part_16/Read/ReadVariableOp with shape: [11, 320]
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_0/dense/kernel/part_17 with shape: []
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_0/dense/kernel/part_17/Read/ReadVariableOp with shape: [11, 320]
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_0/dense/kernel/part_18 with shape: []
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_0/dense/kernel/part_18/Read/ReadVariableOp with shape: [11, 320]
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_0/dense/kernel/part_19 with shape: []
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_0/dense/kernel/part_19/Read/ReadVariableOp with shape: [11, 320]
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_0/dense/kernel/part_20 with shape: []
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_0/dense/kernel/part_20/Read/ReadVariableOp with shape: [10, 320]
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_0/dense/kernel/part_21 with shape: []
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_0/dense/kernel/part_21/Read/ReadVariableOp with shape: [10, 320]
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_0/dense/kernel/part_22 with shape: []
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_0/dense/kernel/part_22/Read/ReadVariableOp with shape: [10, 320]
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_0/dense/kernel/part_23 with shape: []
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_0/dense/kernel/part_23/Read/ReadVariableOp with shape: [10, 320]
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_0/dense/kernel/part_24 with shape: []
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_0/dense/kernel/part_24/Read/ReadVariableOp with shape: [10, 320]
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_0/dense/kernel/part_25 with shape: []
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_0/dense/kernel/part_25/Read/ReadVariableOp with shape: [10, 320]
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_0/dense/kernel/part_26 with shape: []
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_0/dense/kernel/part_26/Read/ReadVariableOp with shape: [10, 320]
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_0/dense/kernel/part_27 with shape: []
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_0/dense/kernel/part_27/Read/ReadVariableOp with shape: [10, 320]
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_0/dense/kernel/part_28 with shape: []
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_0/dense/kernel/part_28/Read/ReadVariableOp with shape: [10, 320]
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_0/dense/kernel/part_29 with shape: []
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_0/dense/kernel/part_29/Read/ReadVariableOp with shape: [10, 320]
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_1/dense/kernel/part_0 with shape: []
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_1/dense/kernel/part_0/Read/ReadVariableOp with shape: [11, 320]
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_1/dense/kernel/part_1 with shape: []
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_1/dense/kernel/part_1/Read/ReadVariableOp with shape: [11, 320]
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_1/dense/kernel/part_2 with shape: []
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_1/dense/kernel/part_2/Read/ReadVariableOp with shape: [11, 320]
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_1/dense/kernel/part_3 with shape: []
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_1/dense/kernel/part_3/Read/ReadVariableOp with shape: [11, 320]
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_1/dense/kernel/part_4 with shape: []
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_1/dense/kernel/part_4/Read/ReadVariableOp with shape: [11, 320]
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_1/dense/kernel/part_5 with shape: []
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_1/dense/kernel/part_5/Read/ReadVariableOp with shape: [11, 320]
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_1/dense/kernel/part_6 with shape: []
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_1/dense/kernel/part_6/Read/ReadVariableOp with shape: [11, 320]
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_1/dense/kernel/part_7 with shape: []
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_1/dense/kernel/part_7/Read/ReadVariableOp with shape: [11, 320]
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_1/dense/kernel/part_8 with shape: []
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_1/dense/kernel/part_8/Read/ReadVariableOp with shape: [11, 320]
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_1/dense/kernel/part_9 with shape: []
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_1/dense/kernel/part_9/Read/ReadVariableOp with shape: [11, 320]
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_1/dense/kernel/part_10 with shape: []
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_1/dense/kernel/part_10/Read/ReadVariableOp with shape: [11, 320]
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_1/dense/kernel/part_11 with shape: []
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_1/dense/kernel/part_11/Read/ReadVariableOp with shape: [11, 320]
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_1/dense/kernel/part_12 with shape: []
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_1/dense/kernel/part_12/Read/ReadVariableOp with shape: [11, 320]
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_1/dense/kernel/part_13 with shape: []
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_1/dense/kernel/part_13/Read/ReadVariableOp with shape: [11, 320]
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_1/dense/kernel/part_14 with shape: []
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_1/dense/kernel/part_14/Read/ReadVariableOp with shape: [11, 320]
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_1/dense/kernel/part_15 with shape: []
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_1/dense/kernel/part_15/Read/ReadVariableOp with shape: [11, 320]
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_1/dense/kernel/part_16 with shape: []
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_1/dense/kernel/part_16/Read/ReadVariableOp with shape: [11, 320]
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_1/dense/kernel/part_17 with shape: []
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_1/dense/kernel/part_17/Read/ReadVariableOp with shape: [11, 320]
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_1/dense/kernel/part_18 with shape: []
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_1/dense/kernel/part_18/Read/ReadVariableOp with shape: [11, 320]
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_1/dense/kernel/part_19 with shape: []
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_1/dense/kernel/part_19/Read/ReadVariableOp with shape: [11, 320]
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_1/dense/kernel/part_20 with shape: []
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_1/dense/kernel/part_20/Read/ReadVariableOp with shape: [10, 320]
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_1/dense/kernel/part_21 with shape: []
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_1/dense/kernel/part_21/Read/ReadVariableOp with shape: [10, 320]
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_1/dense/kernel/part_22 with shape: []
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_1/dense/kernel/part_22/Read/ReadVariableOp with shape: [10, 320]
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_1/dense/kernel/part_23 with shape: []
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_1/dense/kernel/part_23/Read/ReadVariableOp with shape: [10, 320]
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_1/dense/kernel/part_24 with shape: []
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_1/dense/kernel/part_24/Read/ReadVariableOp with shape: [10, 320]
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_1/dense/kernel/part_25 with shape: []
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_1/dense/kernel/part_25/Read/ReadVariableOp with shape: [10, 320]
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_1/dense/kernel/part_26 with shape: []
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_1/dense/kernel/part_26/Read/ReadVariableOp with shape: [10, 320]
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_1/dense/kernel/part_27 with shape: []
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_1/dense/kernel/part_27/Read/ReadVariableOp with shape: [10, 320]
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_1/dense/kernel/part_28 with shape: []
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_1/dense/kernel/part_28/Read/ReadVariableOp with shape: [10, 320]
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_1/dense/kernel/part_29 with shape: []
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_1/dense/kernel/part_29/Read/ReadVariableOp with shape: [10, 320]
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_2/dense/kernel/part_0 with shape: []
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_2/dense/kernel/part_0/Read/ReadVariableOp with shape: [11, 512]
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_2/dense/kernel/part_1 with shape: []
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_2/dense/kernel/part_1/Read/ReadVariableOp with shape: [11, 512]
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_2/dense/kernel/part_2 with shape: []
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_2/dense/kernel/part_2/Read/ReadVariableOp with shape: [11, 512]
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_2/dense/kernel/part_3 with shape: []
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_2/dense/kernel/part_3/Read/ReadVariableOp with shape: [11, 512]
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_2/dense/kernel/part_4 with shape: []
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_2/dense/kernel/part_4/Read/ReadVariableOp with shape: [11, 512]
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_2/dense/kernel/part_5 with shape: []
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_2/dense/kernel/part_5/Read/ReadVariableOp with shape: [11, 512]
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_2/dense/kernel/part_6 with shape: []
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_2/dense/kernel/part_6/Read/ReadVariableOp with shape: [11, 512]
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_2/dense/kernel/part_7 with shape: []
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_2/dense/kernel/part_7/Read/ReadVariableOp with shape: [11, 512]
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_2/dense/kernel/part_8 with shape: []
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_2/dense/kernel/part_8/Read/ReadVariableOp with shape: [11, 512]
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_2/dense/kernel/part_9 with shape: []
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_2/dense/kernel/part_9/Read/ReadVariableOp with shape: [11, 512]
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_2/dense/kernel/part_10 with shape: []
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_2/dense/kernel/part_10/Read/ReadVariableOp with shape: [11, 512]
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_2/dense/kernel/part_11 with shape: []
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_2/dense/kernel/part_11/Read/ReadVariableOp with shape: [11, 512]
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_2/dense/kernel/part_12 with shape: []
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_2/dense/kernel/part_12/Read/ReadVariableOp with shape: [11, 512]
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_2/dense/kernel/part_13 with shape: []
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_2/dense/kernel/part_13/Read/ReadVariableOp with shape: [11, 512]
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_2/dense/kernel/part_14 with shape: []
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_2/dense/kernel/part_14/Read/ReadVariableOp with shape: [11, 512]
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_2/dense/kernel/part_15 with shape: []
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_2/dense/kernel/part_15/Read/ReadVariableOp with shape: [11, 512]
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_2/dense/kernel/part_16 with shape: []
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_2/dense/kernel/part_16/Read/ReadVariableOp with shape: [11, 512]
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_2/dense/kernel/part_17 with shape: []
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_2/dense/kernel/part_17/Read/ReadVariableOp with shape: [11, 512]
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_2/dense/kernel/part_18 with shape: []
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_2/dense/kernel/part_18/Read/ReadVariableOp with shape: [11, 512]
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_2/dense/kernel/part_19 with shape: []
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_2/dense/kernel/part_19/Read/ReadVariableOp with shape: [11, 512]
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_2/dense/kernel/part_20 with shape: []
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_2/dense/kernel/part_20/Read/ReadVariableOp with shape: [10, 512]
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_2/dense/kernel/part_21 with shape: []
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_2/dense/kernel/part_21/Read/ReadVariableOp with shape: [10, 512]
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_2/dense/kernel/part_22 with shape: []
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_2/dense/kernel/part_22/Read/ReadVariableOp with shape: [10, 512]
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_2/dense/kernel/part_23 with shape: []
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_2/dense/kernel/part_23/Read/ReadVariableOp with shape: [10, 512]
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_2/dense/kernel/part_24 with shape: []
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_2/dense/kernel/part_24/Read/ReadVariableOp with shape: [10, 512]
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_2/dense/kernel/part_25 with shape: []
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_2/dense/kernel/part_25/Read/ReadVariableOp with shape: [10, 512]
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_2/dense/kernel/part_26 with shape: []
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_2/dense/kernel/part_26/Read/ReadVariableOp with shape: [10, 512]
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_2/dense/kernel/part_27 with shape: []
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_2/dense/kernel/part_27/Read/ReadVariableOp with shape: [10, 512]
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_2/dense/kernel/part_28 with shape: []
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_2/dense/kernel/part_28/Read/ReadVariableOp with shape: [10, 512]
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_2/dense/kernel/part_29 with shape: []
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_2/dense/kernel/part_29/Read/ReadVariableOp with shape: [10, 512]
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_3/dense/kernel/part_0 with shape: []
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_3/dense/kernel/part_0/Read/ReadVariableOp with shape: [18, 512]
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_3/dense/kernel/part_1 with shape: []
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_3/dense/kernel/part_1/Read/ReadVariableOp with shape: [18, 512]
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_3/dense/kernel/part_2 with shape: []
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_3/dense/kernel/part_2/Read/ReadVariableOp with shape: [17, 512]
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_3/dense/kernel/part_3 with shape: []
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_3/dense/kernel/part_3/Read/ReadVariableOp with shape: [17, 512]
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_3/dense/kernel/part_4 with shape: []
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_3/dense/kernel/part_4/Read/ReadVariableOp with shape: [17, 512]
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_3/dense/kernel/part_5 with shape: []
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_3/dense/kernel/part_5/Read/ReadVariableOp with shape: [17, 512]
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_3/dense/kernel/part_6 with shape: []
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_3/dense/kernel/part_6/Read/ReadVariableOp with shape: [17, 512]
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_3/dense/kernel/part_7 with shape: []
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_3/dense/kernel/part_7/Read/ReadVariableOp with shape: [17, 512]
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_3/dense/kernel/part_8 with shape: []
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_3/dense/kernel/part_8/Read/ReadVariableOp with shape: [17, 512]
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_3/dense/kernel/part_9 with shape: []
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_3/dense/kernel/part_9/Read/ReadVariableOp with shape: [17, 512]
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_3/dense/kernel/part_10 with shape: []
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_3/dense/kernel/part_10/Read/ReadVariableOp with shape: [17, 512]
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_3/dense/kernel/part_11 with shape: []
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_3/dense/kernel/part_11/Read/ReadVariableOp with shape: [17, 512]
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_3/dense/kernel/part_12 with shape: []
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_3/dense/kernel/part_12/Read/ReadVariableOp with shape: [17, 512]
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_3/dense/kernel/part_13 with shape: []
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_3/dense/kernel/part_13/Read/ReadVariableOp with shape: [17, 512]
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_3/dense/kernel/part_14 with shape: []
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_3/dense/kernel/part_14/Read/ReadVariableOp with shape: [17, 512]
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_3/dense/kernel/part_15 with shape: []
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_3/dense/kernel/part_15/Read/ReadVariableOp with shape: [17, 512]
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_3/dense/kernel/part_16 with shape: []
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_3/dense/kernel/part_16/Read/ReadVariableOp with shape: [17, 512]
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_3/dense/kernel/part_17 with shape: []
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_3/dense/kernel/part_17/Read/ReadVariableOp with shape: [17, 512]
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_3/dense/kernel/part_18 with shape: []
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_3/dense/kernel/part_18/Read/ReadVariableOp with shape: [17, 512]
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_3/dense/kernel/part_19 with shape: []
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_3/dense/kernel/part_19/Read/ReadVariableOp with shape: [17, 512]
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_3/dense/kernel/part_20 with shape: []
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_3/dense/kernel/part_20/Read/ReadVariableOp with shape: [17, 512]
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_3/dense/kernel/part_21 with shape: []
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_3/dense/kernel/part_21/Read/ReadVariableOp with shape: [17, 512]
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_3/dense/kernel/part_22 with shape: []
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_3/dense/kernel/part_22/Read/ReadVariableOp with shape: [17, 512]
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_3/dense/kernel/part_23 with shape: []
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_3/dense/kernel/part_23/Read/ReadVariableOp with shape: [17, 512]
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_3/dense/kernel/part_24 with shape: []
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_3/dense/kernel/part_24/Read/ReadVariableOp with shape: [17, 512]
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_3/dense/kernel/part_25 with shape: []
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_3/dense/kernel/part_25/Read/ReadVariableOp with shape: [17, 512]
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_3/dense/kernel/part_26 with shape: []
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_3/dense/kernel/part_26/Read/ReadVariableOp with shape: [17, 512]
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_3/dense/kernel/part_27 with shape: []
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_3/dense/kernel/part_27/Read/ReadVariableOp with shape: [17, 512]
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_3/dense/kernel/part_28 with shape: []
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_3/dense/kernel/part_28/Read/ReadVariableOp with shape: [17, 512]
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_3/dense/kernel/part_29 with shape: []
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_3/dense/kernel/part_29/Read/ReadVariableOp with shape: [17, 512]
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_3/AdjustDepth/projection/kernel/part_0 with shape: []
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_3/AdjustDepth/projection/kernel/part_0/Read/ReadVariableOp with shape: [11, 512]
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_3/AdjustDepth/projection/kernel/part_1 with shape: []
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_3/AdjustDepth/projection/kernel/part_1/Read/ReadVariableOp with shape: [11, 512]
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_3/AdjustDepth/projection/kernel/part_2 with shape: []
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_3/AdjustDepth/projection/kernel/part_2/Read/ReadVariableOp with shape: [11, 512]
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_3/AdjustDepth/projection/kernel/part_3 with shape: []
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_3/AdjustDepth/projection/kernel/part_3/Read/ReadVariableOp with shape: [11, 512]
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_3/AdjustDepth/projection/kernel/part_4 with shape: []
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_3/AdjustDepth/projection/kernel/part_4/Read/ReadVariableOp with shape: [11, 512]
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_3/AdjustDepth/projection/kernel/part_5 with shape: []
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_3/AdjustDepth/projection/kernel/part_5/Read/ReadVariableOp with shape: [11, 512]
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_3/AdjustDepth/projection/kernel/part_6 with shape: []
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_3/AdjustDepth/projection/kernel/part_6/Read/ReadVariableOp with shape: [11, 512]
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_3/AdjustDepth/projection/kernel/part_7 with shape: []
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_3/AdjustDepth/projection/kernel/part_7/Read/ReadVariableOp with shape: [11, 512]
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_3/AdjustDepth/projection/kernel/part_8 with shape: []
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_3/AdjustDepth/projection/kernel/part_8/Read/ReadVariableOp with shape: [11, 512]
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_3/AdjustDepth/projection/kernel/part_9 with shape: []
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_3/AdjustDepth/projection/kernel/part_9/Read/ReadVariableOp with shape: [11, 512]
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_3/AdjustDepth/projection/kernel/part_10 with shape: []
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_3/AdjustDepth/projection/kernel/part_10/Read/ReadVariableOp with shape: [11, 512]
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_3/AdjustDepth/projection/kernel/part_11 with shape: []
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_3/AdjustDepth/projection/kernel/part_11/Read/ReadVariableOp with shape: [11, 512]
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_3/AdjustDepth/projection/kernel/part_12 with shape: []
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_3/AdjustDepth/projection/kernel/part_12/Read/ReadVariableOp with shape: [11, 512]
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_3/AdjustDepth/projection/kernel/part_13 with shape: []
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_3/AdjustDepth/projection/kernel/part_13/Read/ReadVariableOp with shape: [11, 512]
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_3/AdjustDepth/projection/kernel/part_14 with shape: []
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_3/AdjustDepth/projection/kernel/part_14/Read/ReadVariableOp with shape: [11, 512]
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_3/AdjustDepth/projection/kernel/part_15 with shape: []
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_3/AdjustDepth/projection/kernel/part_15/Read/ReadVariableOp with shape: [11, 512]
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_3/AdjustDepth/projection/kernel/part_16 with shape: []
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_3/AdjustDepth/projection/kernel/part_16/Read/ReadVariableOp with shape: [11, 512]
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_3/AdjustDepth/projection/kernel/part_17 with shape: []
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_3/AdjustDepth/projection/kernel/part_17/Read/ReadVariableOp with shape: [11, 512]
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_3/AdjustDepth/projection/kernel/part_18 with shape: []
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_3/AdjustDepth/projection/kernel/part_18/Read/ReadVariableOp with shape: [11, 512]
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_3/AdjustDepth/projection/kernel/part_19 with shape: []
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_3/AdjustDepth/projection/kernel/part_19/Read/ReadVariableOp with shape: [11, 512]
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_3/AdjustDepth/projection/kernel/part_20 with shape: []
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_3/AdjustDepth/projection/kernel/part_20/Read/ReadVariableOp with shape: [10, 512]
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_3/AdjustDepth/projection/kernel/part_21 with shape: []
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_3/AdjustDepth/projection/kernel/part_21/Read/ReadVariableOp with shape: [10, 512]
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_3/AdjustDepth/projection/kernel/part_22 with shape: []
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_3/AdjustDepth/projection/kernel/part_22/Read/ReadVariableOp with shape: [10, 512]
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_3/AdjustDepth/projection/kernel/part_23 with shape: []
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_3/AdjustDepth/projection/kernel/part_23/Read/ReadVariableOp with shape: [10, 512]
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_3/AdjustDepth/projection/kernel/part_24 with shape: []
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_3/AdjustDepth/projection/kernel/part_24/Read/ReadVariableOp with shape: [10, 512]
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_3/AdjustDepth/projection/kernel/part_25 with shape: []
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_3/AdjustDepth/projection/kernel/part_25/Read/ReadVariableOp with shape: [10, 512]
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_3/AdjustDepth/projection/kernel/part_26 with shape: []
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_3/AdjustDepth/projection/kernel/part_26/Read/ReadVariableOp with shape: [10, 512]
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_3/AdjustDepth/projection/kernel/part_27 with shape: []
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_3/AdjustDepth/projection/kernel/part_27/Read/ReadVariableOp with shape: [10, 512]
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_3/AdjustDepth/projection/kernel/part_28 with shape: []
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_3/AdjustDepth/projection/kernel/part_28/Read/ReadVariableOp with shape: [10, 512]
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_3/AdjustDepth/projection/kernel/part_29 with shape: []
[notice ] ofxTensorFlow2: EncoderDNN/DNN/ResidualHidden_3/AdjustDepth/projection/kernel/part_29/Read/ReadVariableOp with shape: [10, 512]
[notice ] ofxTensorFlow2: StatefulPartitionedCall with shape: []
[notice ] ofxTensorFlow2: Const with shape: []
[notice ] ofxTensorFlow2: serving_default_inputs with shape: [-1]
[notice ] ofxTensorFlow2: StatefulPartitionedCall_1 with shape: [-1, 512]
[notice ] ofxTensorFlow2: saver_filename with shape: []
[notice ] ofxTensorFlow2: StatefulPartitionedCall_2 with shape: []
[notice ] ofxTensorFlow2: StatefulPartitionedCall_3 with shape: []
[notice ] ofxTensorFlow2: ============ End ==============

@Jonathhhan
Copy link
Contributor Author

Jonathhhan commented Jun 14, 2022 •
edited

Actually there is a warning if I run the universal sentence encoder model (happens with other text models too): Failed to place the graph without changing the devices of some resources. Some of the operations (that had to be colocated with resource generating operations) are not supported on the resources' devices.

2022-06-15 01:54:24.035736: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2022-06-15 01:54:24.341625: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1532] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 12287 MB memory:  -> device: 0, name: NVIDIA GeForce RTX 3090, pci bus id: 0000:01:00.0, compute capability: 8.6
2022-06-15 01:54:24.342986: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1532] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 12287 MB memory:  -> device: 0, name: NVIDIA GeForce RTX 3090, pci bus id: 0000:01:00.0, compute capability: 8.6
2022-06-15 01:54:24.355258: I tensorflow/cc/saved_model/reader.cc:43] Reading SavedModel from: data\model3
2022-06-15 01:54:24.398104: I tensorflow/cc/saved_model/reader.cc:81] Reading meta graph with tags { serve }
2022-06-15 01:54:24.398212: I tensorflow/cc/saved_model/reader.cc:122] Reading SavedModel debug info (if present) from: data\model3
2022-06-15 01:54:24.398526: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1532] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 12287 MB memory:  -> device: 0, name: NVIDIA GeForce RTX 3090, pci bus id: 0000:01:00.0, compute capability: 8.6
2022-06-15 01:54:24.549797: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:354] MLIR V1 optimization pass is not enabled
2022-06-15 01:54:24.609551: I tensorflow/cc/saved_model/loader.cc:228] Restoring SavedModel bundle.
2022-06-15 01:54:29.123234: I tensorflow/cc/saved_model/loader.cc:212] Running initialization op on SavedModel bundle at path: data\model3
2022-06-15 01:54:30.281562: W tensorflow/core/common_runtime/colocation_graph.cc:1200] Failed to place the graph without changing the devices of some resources. Some of the operations (that had to be colocated with resource generating operations) are not supported on the resources' devices. Current candidate devices are [
  /job:localhost/replica:0/task:0/device:CPU:0].
See below for details of this colocation group:
Colocation Debug Info:
Colocation group had the following types and supported devices:
Root Member(assigned_device_name_index_=-1 requested_device_name_='/job:localhost/replica:0/task:0/device:GPU:0' assigned_device_name_='' resource_device_name_='/job:localhost/replica:0/task:0/device:GPU:0' supported_device_types_=[CPU] possible_devices_=[]
HashTableV2: CPU
LookupTableImportV2: CPU

Colocation members, user-requested devices, and framework assigned devices, if any:
  StatefulPartitionedCall/StatefulPartitionedCall/text_preprocessor/hash_table (HashTableV2) /job:localhost/replica:0/task:0/device:GPU:0
  StatefulPartitionedCall/StatefulPartitionedCall/text_preprocessor/key_value_init/LookupTableImportV2 (LookupTableImportV2) /job:localhost/replica:0/task:0/device:GPU:0

2022-06-15 01:54:30.281832: W tensorflow/core/common_runtime/colocation_graph.cc:1200] Failed to place the graph without changing the devices of some resources. Some of the operations (that had to be colocated with resource generating operations) are not supported on the resources' devices. Current candidate devices are [
  /job:localhost/replica:0/task:0/device:CPU:0].
See below for details of this colocation group:
Colocation Debug Info:
Colocation group had the following types and supported devices:
Root Member(assigned_device_name_index_=-1 requested_device_name_='/job:localhost/replica:0/task:0/device:GPU:0' assigned_device_name_='' resource_device_name_='/job:localhost/replica:0/task:0/device:GPU:0' supported_device_types_=[CPU] possible_devices_=[]
HashTableV2: CPU
LookupTableImportV2: CPU

Colocation members, user-requested devices, and framework assigned devices, if any:
  StatefulPartitionedCall/StatefulPartitionedCall/text_preprocessor_1/hash_table (HashTableV2) /job:localhost/replica:0/task:0/device:GPU:0
  StatefulPartitionedCall/StatefulPartitionedCall/text_preprocessor_1/key_value_init/LookupTableImportV2 (LookupTableImportV2) /job:localhost/replica:0/task:0/device:GPU:0

2022-06-15 01:54:31.547040: I tensorflow/cc/saved_model/loader.cc:301] SavedModel load for tags { serve }; Status: success: OK. Took 7191761 microseconds.

This is basically what I would like with ofxTensoflow2: https://www.youtube.com/watch?v=YjRhIpfz5ms

@Jonathhhan Jonathhhan changed the title word2vec example? word2vec / universal sentence encoder example? Jun 15, 2022
@Jonathhhan
Copy link
Contributor Author

Jonathhhan commented Jun 16, 2022 •
edited

I made a universal sentence encoder example: https://github.com/Jonathhhan/ofxTensorFlow2/tree/universal-sentence-encoder/example_universal_sentence_encoder
It uses this model: https://tfhub.dev/google/universal-sentence-encoder/4
This one is working, too: https://tfhub.dev/google/universal-sentence-encoder-large/5

@Jonathhhan
Copy link
Contributor Author

I updated the example, its a basic version of my "Montageautomat": https://github.com/Jonathhhan/ofxTensorFlow2/tree/universal-sentence-encoder/example_universal_sentence_encoder_video_2

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@bytosaur @Jonathhhan