TensorFlow logs not showing up on CloudWatch or console

[cgarciae]

A recent changed on sagemaker caused tensorflow logs not showing up on the console and cloudwatch. This makes a lot of tasks very difficult including hyper parameter tunning. I've tested on my own code and the abalone example: no tf logs shown.

[ChoiByungWook]

Hello,

Thanks for using SageMaker!

We are aware of this issue and are working towards releasing a fix.

We will provide an update when the fix has been released.

Thanks!

[ChoiByungWook]

@cgarciae Thank you for bringing up this issue to our attention.

We have released a fix, please let us know if it works for you.

Thanks!

[ChoiByungWook]

Closing, please open another issue if this is still a problem. Thanks!

[tspeterkim]

@cgarciae I'm still running into the same issue. are you using
tf.logging.info() to send your logs? Thanks

[Nimrods]

@peterkim95 I was able to see logs (training info like loss) in my pycharm console after setting the
tf.logging.set_verbosity(tf.logging.INFO) in my entrypoint script