make_iterator_one_shot results in IndexError: list index out of range on first call (only)

[skeydan]

Reminding of rstudio/tensorflow#272, but with current master.
This only happens on first call, second execution works:

reticulate::use_condaenv("tf-master-1215")
library(tensorflow)
tfe_enable_eager_execution()
library(tfdatasets)
dataset <- tensor_slices_dataset(matrix(1:2))
iter <- dataset %>% make_iterator_one_shot()

> iter <- dataset %>% make_iterator_one_shot()
Error in py_call_impl(callable, dots$args, dots$keywords) : 
  IndexError: list index out of range

Detailed traceback: 
  File "/home/key/anaconda3/envs/tf-master-1215/lib/python3.6/site-packages/tensorflow/python/util/deprecation.py", line 314, in new_func
    _call_location(), decorator_utils.get_qualified_name(func),
  File "/home/key/anaconda3/envs/tf-master-1215/lib/python3.6/site-packages/tensorflow/python/util/deprecation.py", line 103, in _call_location
    frame = stack[-4 if outer else -3]

[skeydan]

Hi @javierluraschi, could you take a look and see if this should have been eliminated by your fix to tensorflow/tensorflow#23388?

W.r.t. rstudio/tensorflow#272 I tested that weeks ago, but this case might be slightly different.

Here I think it's the deprecation warning that's issued the first time you call something deprecated in a session,- then all further calls go through "undecorated".
So I think this could affect us whenever we call a deprecated method.

BTW this can be tested with a nightly install of master.

[skeydan]

I think we have no choice than changing all methods that use deprecated code (there are many of them, tests get aborted b/c too many errors).
I've added one specific workaround here: #22

But that can only be temporary... Their real updating advice is:

Use for ... in dataset: to iterate over a dataset.

We need to think how to do that on our side ...

[skeydan]

Just noting this for some time later:

• deprecation warnings are originating from tensorflow/python/data/ops/dataset_ops.py
• we could probably use dataset$reduce to mimick Python's for _ in dataset (see

https://github.com/tensorflow/docs/blob/master/site/en/r2/guide/effective_tf2.md

which describes this as being the action automatically taken by autograph)

[skeydan]

R-side workaround in

rstudio/tensorflow#287

but I'll leave it open until I've created an issue in TF

[skeydan]

tensorflow/tensorflow#25067

[skeydan]

It's already fixed in TF master, as I can confirm locally removing our workaround and doing

> library(tensorflow)
> tf_config()
TensorFlow v1.13.0-dev20190122 (~/anaconda3/envs/r-tensorflow/lib/python3.6/site-packages/tensorflow)
Python v3.6 (~/anaconda3/envs/r-tensorflow/bin/python)
> library(tfdatasets)
> dataset <- tensor_slices_dataset(matrix(1:2))
> iter <- dataset %>% make_iterator_one_shot()
WARNING:tensorflow:From /home/key/anaconda3/envs/r-tensorflow/lib/python3.6/site-packages/tensorflow/python/data/ops/dataset_ops.py:1730: DatasetV1.make_one_shot_iterator (from tensorflow.python.data.ops.dataset_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use `for ... in dataset:` to iterate over a dataset. If using `tf.estimator`, return the `Dataset` object directly from your input function. As a last resort, you can use `tf.compat.v1.data.make_one_shot_iterator(dataset)`.
> 

Now we just need to see if it still makes its way into 1.13 (I've asked)

[skeydan]

seems like it made it into 1.13

"Added bounds checking to printing deprecation warnings."