Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

tqdm stuck strangely when tracking progress for a tensorflow.RaggedTensor generator #951

Open
5 of 8 tasks
atodniAr opened this issue Apr 27, 2020 · 0 comments
Open
5 of 8 tasks

Comments

@atodniAr
Copy link

  • I have marked all applicable categories:
    • exception-raising bug
    • visual output bug
    • documentation request (i.e. "X is missing from the documentation." If instead I want to ask "how to use X?" I understand StackOverflow#tqdm is more appropriate)
    • new feature request
  • I have visited the source website, and in particular
    read the known issues
  • I have searched through the issue tracker for duplicates
  • I have mentioned version numbers, operating system and
    environment, where applicable:
    import tqdm, sys
    print(tqdm.__version__, sys.version, sys.platform)

Version: tqdm 4.45.0
Python: 3.7.0
System platform: [GCC 7.3.0] linux

I have a user defined generator, which generates a tuple (tf.RaggedTensor, bool) per call. It works with no problem when I call it with:

g = hierarchical_random_sequence() # generator
_train_temp = [next(g) for i in range(10000)] # testing

However, it stucks with tqdm. I've retried several times with no luck. It never raised an error (or I didn't wait for long enough), just stuck for several mins. I have to keyboard interrupt it there after.

100%|█████████▉| 9978/10000 [00:25<00:00, 317.95it/s]
---------------------------------------------------------------------------
KeyboardInterrupt                         Traceback (most recent call last)
<ipython-input-426-1ea411223d67> in <module>
      4 
      5 g = hierarchical_random_sequence()
----> 6 _train_temp = [next(g) for i in tqdm(range(sample_count))]
      7 #x_train = tf.ragged.stack([i[0] for i in _train_temp])
      8 #y_train = tf.reshape(tf.constant([i[1] for i in _train_temp]),[-1,1])

<ipython-input-426-1ea411223d67> in <listcomp>(.0)
      4 
      5 g = hierarchical_random_sequence()
----> 6 _train_temp = [next(g) for i in tqdm(range(sample_count))]
      7 #x_train = tf.ragged.stack([i[0] for i in _train_temp])
      8 #y_train = tf.reshape(tf.constant([i[1] for i in _train_temp]),[-1,1])

/data/env/battery_dl/lib/python3.7/site-packages/tqdm/std.py in __iter__(self)
   1178             self.n = n
   1179             self.miniters = miniters
-> 1180             self.close()
   1181 
   1182     def update(self, n=1):

/data/env/battery_dl/lib/python3.7/site-packages/tqdm/std.py in close(self)
   1266         # decrement instance pos and remove from internal set
   1267         pos = abs(self.pos)
-> 1268         self._decr_instances(self)
   1269 
   1270         # GUI mode

/data/env/battery_dl/lib/python3.7/site-packages/tqdm/std.py in _decr_instances(cls, instance)
    570             if not cls._instances and cls.monitor:
    571                 try:
--> 572                     cls.monitor.exit()
    573                     del cls.monitor
    574                 except AttributeError:  # pragma: nocover

/data/env/battery_dl/lib/python3.7/site-packages/tqdm/_monitor.py in exit(self)
     51         self.was_killed.set()
     52         if self is not current_thread():
---> 53             self.join()
     54         return self.report()
     55 

/data/env/battery_dl/lib/python3.7/threading.py in join(self, timeout)
   1030 
   1031         if timeout is None:
-> 1032             self._wait_for_tstate_lock()
   1033         else:
   1034             # the behavior of a negative timeout isn't documented, but

/data/env/battery_dl/lib/python3.7/threading.py in _wait_for_tstate_lock(self, block, timeout)
   1046         if lock is None:  # already determined that the C code is done
   1047             assert self._is_stopped
-> 1048         elif lock.acquire(block, timeout):
   1049             lock.release()
   1050             self._stop()

KeyboardInterrupt:

I'm not sure what kind of interaction is happening between tqdm and tensorflow, or just tqdm itself. The same operation works without tqdm, so memory should not be the issue.

My tensorflow version is 2.2.0rc3 fyi.

@atodniAr atodniAr changed the title tqdm stuck strangely when tracking progress for a tensorflow.Raggedtensor generator tqdm stuck strangely when tracking progress for a tensorflow.RaggedTensor generator Apr 27, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant