You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
it means that you have very long sentences in your datasets which consumer a lot of memory during baching. you need to summarize your dataset line by line using an intelligent summarize like textrank which is supervised --doesn't require training. Then try this summarized dataset ..it can even be 10 GB and you won't have any problem.
Just to point out that use
tf.convert_to_tensor -> tf.train.slice_input_producer -> tf.train.shuffle_batch
will meet an error
if dataset is too large
The text was updated successfully, but these errors were encountered: