Different results between Safari and Chrome

[oxxie]

TensorFlow.js version

0.11.6

Browser version

Safari 11.1.1 / Chrome 67.0.3396.79

Describe the problem or feature request

While running the exact same script of a simple XOR-problem on both Safari and Chrome, both browsers display very different losses after Model.fit() was performed multiple times. While Chrome keeps producing smaller losses (around 0.001), Safari gets stuck at a fixed loss (around 0.25).

Suggestion: could this be a precision issue on the Safari platform?

Code to reproduce the bug / link to feature request

Nothing fancy here, but just to be sure:

async function trainModel() {
  const h = await model.fit(trainingData.X, trainingData.Y, trainingConfig);
  console.log(h.history.loss[0]);
}

[bileschi]

Thanks @oxxie ,

Is it possible to share a very small test case, including possibly simulated or hard-coded data, that replicates the problem?

[oxxie]

Sure, here goes:

const model = tf.sequential();

const trainingData = {
  x: tf.tensor2d([[0, 0], [0, 1], [1, 0], [1, 1]]),
  y: tf.tensor2d([[0], [1], [1], [0]])
}

async function trainModel() {
  const h = await model.fit(trainingData.x, trainingData.y);
  console.log(h.history.loss[0]);
}

model.add(tf.layers.dense({units: 2, activation: 'sigmoid', inputShape: [2]}));
model.add(tf.layers.dense({units: 1, activation: 'sigmoid'}));
model.compile({optimizer: tf.train.sgd(0.1), loss: tf.losses.meanSquaredError});

setInterval(trainModel, 100);

In Chrome, this will result in ever-decreasing losses, while in Safari the loss will 'get stuck' quite fast, around 0.25 in my case.

Also, I discovered today that switching the optimizer to tf.train.adam(0.1) will result in NaN losses in Safari... I didn't check any other optimizers though.

Hope this helps, let me know if you need anything else!

[nsthorat]

This should be fixed now, can you verify @oxxie?

In iOS there may still be some small differences because we use float16 under the hood instead of float32, but they shouldn't be huge differences.

[oxxie]

Verified with TensorFlow.js version 0.13.3. Losses are now consistent between Chrome and Safari 😀 Thanks for looking into it and the fix! Will close this issue now.

[yqtianust]

Hi, may I know the root cause of this issue? Thanks.