test: Gradient descent optimizer tests #1184

novikov-alexander · 2023-09-27T13:18:08Z

No description provided.

novikov-alexander · 2023-09-27T13:19:53Z

test/TensorFlowNET.UnitTest/PythonTest.cs

+            }
+        }
+
+        public void assertAllCloseAccordingToType<T>(


@Wanglongzhi2001 Does it make sense not to duplicate all assertion code but combine all in one assembly?

src/TensorFlowNET.Core/Variables/variables.py.cs

Oceania2018 · 2023-09-29T19:20:45Z

 Failed TestBasic [1 s]
  Error Message:
   Test method Tensorflow.Keras.UnitTest.Optimizers.GradientDescentOptimizerTest.TestBasic threw exception: 
System.NullReferenceException: Object reference not set to an instance of an object.
  Stack Trace:
      at TensorFlowNET.UnitTest.PythonTest.evaluate[T](Tensor tensor) in D:\a\TensorFlow.NET\TensorFlow.NET\test\TensorFlowNET.UnitTest\PythonTest.cs:line 245
   at Tensorflow.Keras.UnitTest.Optimizers.GradientDescentOptimizerTest.TestBasicGeneric[T]() in D:\a\TensorFlow.NET\TensorFlow.NET\test\TensorFlowNET.UnitTest\Training\GradientDescentOptimizerTests.cs:line 40
   at Tensorflow.Keras.UnitTest.Optimizers.GradientDescentOptimizerTest.TestBasic() in D:\a\TensorFlow.NET\TensorFlow.NET\test\TensorFlowNET.UnitTest\Training\GradientDescentOptimizerTests.cs:line 62

…nto alnovi/optimizer_tests

test/TensorFlowNET.UnitTest/PythonTest.cs

novikov-alexander · 2023-11-07T19:32:51Z

test/TensorFlowNET.UnitTest/PythonTest.cs

@@ -196,17 +245,25 @@ public T evaluate<T>(Tensor tensor)
            //    return self._eval_helper(tensors)
            //  else:
            {
-                var sess = tf.Session();
+                var sess = tf.get_default_session();


@Wanglongzhi2001 The same functions exist in another assembly.
Should we generalize them in one common assembly in another PR?

I think so, if you would like to.

novikov-alexander · 2023-11-07T19:33:58Z

test/TensorFlowNET.UnitTest/PythonTest.cs

+                {
+                    throw new ValueError(@"The config used to get the cached session is 
+                                           different than the one that was used to create the 
+                                           session. Maybe create a new session with 


@Wanglongzhi2001 this code is also copy pasted from my previous PR because that's how test architecture implemented right now. Should I move it to one common assembly?

I think moving to a common namespace is a good idea if it could be re-used by other modules. Please open an another PR if you would like to do that.

novikov-alexander · 2023-11-07T21:58:24Z

test/TensorFlowNET.UnitTest/PythonTest.cs

@@ -196,17 +245,25 @@ public T evaluate<T>(Tensor tensor)
            //    return self._eval_helper(tensors)
            //  else:
            {
-                var sess = tf.Session();
+                var sess = tf.get_default_session();
                var ndarray = tensor.eval(sess);


@Wanglongzhi2001 what is the reason tensor.eval used there?
Maybe replace by sess.run(tensor) how it is made in original tests?

I think it's to map the tensor to a node in the session (graph).

novikov-alexander · 2023-11-08T04:10:59Z

test/TensorFlowNET.UnitTest/Training/GradientDescentOptimizerTests.cs

+                self.assertAllCloseAccordingToType(
+                    new[] { 3.0 - 3.0 * 0.01, 4.0 - 3.0 * 0.01 },
+                    self.evaluate<T[]>(var1));
+                // TODO: self.assertEqual(0, len(optimizer.variables()));


@Wanglongzhi2001 Do I understand correctly that this one is not applicable for TensorFlow.NET?

novikov-alexander · 2023-11-08T04:19:26Z

test/TensorFlowNET.UnitTest/Training/GradientDescentOptimizerTests.cs

+                var var1 = tf.Variable(new[] { 3.0, 4.0 }, dtype: dtype);
+                var grads0 = tf.constant(new[] { 0.1, 0.1 }, dtype: dtype);
+                var grads1 = tf.constant(new[] { 0.01, 0.01 }, dtype: dtype);
+                var optimizer = tf.train.GradientDescentOptimizer(3.0f);


@Wanglongzhi2001 TensorFlow can accept lambda in this constructor and there's a test testing it.
Is it intentionally that TensorFlow.NET doesn't support that interface?

I don't think so, it's more like a compromise to the early fast-development. Therefore only a smallest and most-used implementation was provided. I think it's feasible to support it.

Wanglongzhi2001 · 2023-11-08T04:53:49Z

@novikov-alexander Please fix the ci error.

…nto alnovi/optimizer_tests

…lexander/TensorFlow.NET into alnovi/optimizer_tests

novikov-alexander · 2023-11-09T22:26:53Z

@novikov-alexander Please fix the ci error.

Not clear how they interfer. I'll try to debug.

passing

novikov-alexander · 2023-11-10T15:01:28Z

@Wanglongzhi2001 That's not clear.. I'm afraid that LinearRegression is flaky one. I just put my tests near and it caused some fail inside. But there are no visible reasons. Moreover it stopped to fail when I changed namespace of my tests.

novikov-alexander · 2023-11-10T15:09:11Z

All checks have passed

Wanglongzhi2001 · 2023-11-10T16:43:04Z

@Oceania2018 LGTM.

AsakusaRinne

Thank you a lot for this contribution! All things seems to be well done. If you'd like to change anything mentioned in the previous comments, please open a new PR. :)

AsakusaRinne · 2023-11-10T19:01:48Z

test/TensorFlowNET.UnitTest/PythonTest.cs

@@ -196,17 +245,25 @@ public T evaluate<T>(Tensor tensor)
            //    return self._eval_helper(tensors)
            //  else:
            {
-                var sess = tf.Session();
+                var sess = tf.get_default_session();


I think so, if you would like to.

AsakusaRinne · 2023-11-10T19:03:55Z

test/TensorFlowNET.UnitTest/PythonTest.cs

@@ -196,17 +245,25 @@ public T evaluate<T>(Tensor tensor)
            //    return self._eval_helper(tensors)
            //  else:
            {
-                var sess = tf.Session();
+                var sess = tf.get_default_session();
                var ndarray = tensor.eval(sess);


I think it's to map the tensor to a node in the session (graph).

AsakusaRinne · 2023-11-10T19:08:37Z

test/TensorFlowNET.UnitTest/PythonTest.cs

+                {
+                    throw new ValueError(@"The config used to get the cached session is 
+                                           different than the one that was used to create the 
+                                           session. Maybe create a new session with 


I think moving to a common namespace is a good idea if it could be re-used by other modules. Please open an another PR if you would like to do that.

AsakusaRinne · 2023-11-10T19:12:56Z

test/TensorFlowNET.UnitTest/Training/GradientDescentOptimizerTests.cs

+                var var1 = tf.Variable(new[] { 3.0, 4.0 }, dtype: dtype);
+                var grads0 = tf.constant(new[] { 0.1, 0.1 }, dtype: dtype);
+                var grads1 = tf.constant(new[] { 0.01, 0.01 }, dtype: dtype);
+                var optimizer = tf.train.GradientDescentOptimizer(3.0f);


I don't think so, it's more like a compromise to the early fast-development. Therefore only a smallest and most-used implementation was provided. I think it's feasible to support it.

gradient descent tests

2121079

novikov-alexander commented Sep 27, 2023

View reviewed changes

src/TensorFlowNET.Core/Variables/variables.py.cs Outdated Show resolved Hide resolved

Oceania2018 reviewed Sep 29, 2023

View reviewed changes

src/TensorFlowNET.Core/Variables/variables.py.cs Show resolved Hide resolved

Oceania2018 assigned novikov-alexander and Wanglongzhi2001 Sep 29, 2023

novikov-alexander added 2 commits November 7, 2023 16:07

Merge branch 'master' of https://github.com/SciSharp/TensorFlow.NET i…

c32d153

…nto alnovi/optimizer_tests

tests are passing

2a377e2

novikov-alexander marked this pull request as ready for review November 7, 2023 19:29

novikov-alexander commented Nov 7, 2023

View reviewed changes

test/TensorFlowNET.UnitTest/PythonTest.cs Show resolved Hide resolved

novikov-alexander commented Nov 7, 2023

View reviewed changes

novikov-alexander commented Nov 8, 2023

View reviewed changes

novikov-alexander and others added 5 commits November 8, 2023 15:16

small fixes

f7b8dba

learning rate test

c906f46

Merge branch 'master' of https://github.com/SciSharp/TensorFlow.NET i…

b1972a8

…nto alnovi/optimizer_tests

Merge branch 'SciSharp:master' into alnovi/optimizer_tests

2a5d0f7

Merge branch 'alnovi/optimizer_tests' of https://github.com/novikov-a…

e6c7c79

…lexander/TensorFlow.NET into alnovi/optimizer_tests

novikov-alexander force-pushed the alnovi/optimizer_tests branch 3 times, most recently from 9c8d624 to e6c7c79 Compare November 9, 2023 22:19

novikov-alexander force-pushed the alnovi/optimizer_tests branch 3 times, most recently from 239e8de to 178296b Compare November 10, 2023 03:57

test ci

149caae

novikov-alexander force-pushed the alnovi/optimizer_tests branch from 178296b to 149caae Compare November 10, 2023 04:13

new graph

2cb5fd6

Oceania2018 approved these changes Nov 10, 2023

View reviewed changes

novikov-alexander force-pushed the alnovi/optimizer_tests branch from 5196950 to 3c53b8b Compare November 10, 2023 14:01

ci test

09d466d

novikov-alexander force-pushed the alnovi/optimizer_tests branch from 3c53b8b to 09d466d Compare November 10, 2023 14:23

novikov-alexander changed the title ~~Gradient descent optimizer tests~~ test: Gradient descent optimizer tests Nov 10, 2023

correct namespace

c5b4928

passing

novikov-alexander force-pushed the alnovi/optimizer_tests branch from b104dc2 to c5b4928 Compare November 10, 2023 14:59

AsakusaRinne approved these changes Nov 10, 2023

View reviewed changes

AsakusaRinne merged commit d020897 into SciSharp:master Nov 10, 2023
3 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

test: Gradient descent optimizer tests #1184

test: Gradient descent optimizer tests #1184

novikov-alexander commented Sep 27, 2023

novikov-alexander Sep 27, 2023

Oceania2018 commented Sep 29, 2023

novikov-alexander Nov 7, 2023

AsakusaRinne Nov 10, 2023

novikov-alexander Nov 7, 2023

AsakusaRinne Nov 10, 2023

novikov-alexander Nov 7, 2023

AsakusaRinne Nov 10, 2023

novikov-alexander Nov 8, 2023

novikov-alexander Nov 8, 2023

AsakusaRinne Nov 10, 2023

Wanglongzhi2001 commented Nov 8, 2023

novikov-alexander commented Nov 9, 2023

novikov-alexander commented Nov 10, 2023

novikov-alexander commented Nov 10, 2023

Wanglongzhi2001 commented Nov 10, 2023

AsakusaRinne left a comment

AsakusaRinne Nov 10, 2023

AsakusaRinne Nov 10, 2023

AsakusaRinne Nov 10, 2023

AsakusaRinne Nov 10, 2023

test: Gradient descent optimizer tests #1184

test: Gradient descent optimizer tests #1184

Conversation

novikov-alexander commented Sep 27, 2023

Choose a reason for hiding this comment

Oceania2018 commented Sep 29, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Wanglongzhi2001 commented Nov 8, 2023

novikov-alexander commented Nov 9, 2023

novikov-alexander commented Nov 10, 2023

novikov-alexander commented Nov 10, 2023

Wanglongzhi2001 commented Nov 10, 2023

AsakusaRinne left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment