Cost(loss) result using placeholder vs. primitive list

[syenpark]

I spotted cost result of 'lab-02-2-linear_regression_feed.py' is slightly different in using placeholder and using a python list.

x_train = [1, 2, 3]
y_train = [1, 2, 3]

# Our hypothesis XW+b
hypothesis = x_train * W + b

# cost/loss function
cost = tf.reduce_mean(tf.square(hypothesis - y_train))

0 2.82329 [ 2.12867713] [-0.85235667]
20 0.190351 [ 1.53392804] [-1.05059612]
...
1980 1.32962e-05 [ 1.00423515] [-0.00962736]
2000 1.20761e-05 [ 1.00403607] [-0.00917497]

[ 5.0110054]
[ 2.50091505]
[ 1.49687922 3.50495124]

X = tf.placeholder(tf.float32, shape=[None])
Y = tf.placeholder(tf.float32, shape=[None])

# Our hypothesis XW+b
hypothesis = X * W + b

# cost/loss function
cost = tf.reduce_mean(tf.square(hypothesis - Y))

sess.run([cost, W, b, train], feed_dict={X: [1, 2, 3], Y: [1, 2, 3]})

0 3.52408 [ 2.12867713] [-0.85235667]
20 0.197499 [ 1.53392804] [-1.05059612]
1960 1.47111e-05 [ 1.004444] [-0.01010205]
1980 1.3361e-05 [ 1.00423515] [-0.00962736]
...
2000 1.21343e-05 [ 1.00403607] [-0.00917497]

[ 5.0110054]
[ 2.50091505]
[ 1.49687922 3.50495124]

Hypothesis, W, and b are the same, while cost result is different even with those same values. I suppose this cost result should be equal regardless of node type, so that I cannot understand this discrepancy in cost and conformability in others in spite of that. Did I something miss?

[kkweon]

W and b are learnable parameters and these values at optimal or at the final step are not unique, so it is normal that values are slightly different.

Oops nvm, I read wrong.
I was not able to reproduce this error.
Mine shows the same.

# list version
0 137.324 [-0.00311102] [ 0.]
100 0.0745313 [ 2.1351583] [ 2.42115235]
200 0.0168967 [ 2.0603478] [ 2.71034646]
300 0.00242139 [ 2.02283072] [ 2.89033508]
400 0.000215083 [ 2.00680494] [ 2.96731591]
500 1.19252e-05 [ 2.00160241] [ 2.99230409]
600 4.07653e-07 [ 2.00029635] [ 2.99857688]
700 8.39337e-09 [ 2.00004268] [ 2.99979568]
800 1.0849e-10 [ 2.00000477] [ 2.99997711]
**900 1.0784e-11 [ 2.00000167] [ 2.99999261]**
==========
# placeholder
0 136.361 [ 0.00615465] [ 0.]
100 0.0755595 [ 2.13604784] [ 2.41677833]
200 0.0170177 [ 2.06055903] [ 2.70930576]
300 0.00241298 [ 2.02279139] [ 2.89052629]
400 0.000211428 [ 2.00674629] [ 2.96759462]
500 1.1529e-05 [ 2.00157547] [ 2.99243259]
600 3.8709e-07 [ 2.00028872] [ 2.9986136]
700 7.72112e-09 [ 2.00004077] [ 2.99980354]
800 8.94228e-11 [ 2.00000453] [ 2.99997878]
**900 1.0784e-11 [ 2.00000167] [ 2.99999261]**

[syenpark]

@kkweon, thanks for your kindly help! I will double check whether I missed or not. Should others also have the same problem, please let me know!

+++
I suppose it is related to step or feed_dict timing(?) instead of placeholder vs list.

import tensorflow as tf

# Model parameters
W = tf.Variable([.3], tf.float32)
b = tf.Variable([-.3], tf.float32)

# Model input and output
x = tf.placeholder(tf.float32)
y = tf.placeholder(tf.float32)

linear_model = x * W + b

# cost/loss function
loss = tf.reduce_sum(tf.square(linear_model - y))  # sum of the squares

# optimizer
optimizer = tf.train.GradientDescentOptimizer(0.01)
train = optimizer.minimize(loss)

# training data
x_train = [1, 2, 3, 4]
y_train = [0, -1, -2, -3]

# training loop
init = tf.global_variables_initializer()
sess = tf.Session()
sess.run(init)  # reset values to wrong
for i in range(1000):
    curr_W, curr_b, curr_loss, _ = sess.run([W, b, loss, train], {x: x_train, y: y_train})
print("W: %s b: %s loss: %s" % (curr_W, curr_b, curr_loss))

# evaluate training accuracy
curr_W, curr_b, curr_loss = sess.run([W, b, loss], {x: x_train, y: y_train})
print("W: %s b: %s loss: %s" % (curr_W, curr_b, curr_loss))

W: [-0.9999969] b: [ 0.99999082] loss: 5.77707e-11
W: [-0.9999969] b: [ 0.99999082] loss: 5.69997e-11

Please explain the reason above results' loss functions are different? I thought below one is just for evaluating, not training one more.

[github-actions]

Stale issue message