RuntimeError: Set changed size during iteration #481

utkonos · 2017-11-19T09:23:48Z

tqdm is throwing a set changed size during iteration exception intermittently:

Exception in thread Thread-7:
Traceback (most recent call last):
  File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
    self.run()
  File "/home/vagrant/regex/lib/python3.6/site-packages/tqdm/_tqdm.py", line 144, in run
    for instance in self.tqdm_cls._instances:
  File "/usr/lib/python3.6/_weakrefset.py", line 60, in __iter__
    for itemref in self.data:
RuntimeError: Set changed size during iteration

The text was updated successfully, but these errors were encountered:

utkonos · 2017-11-19T09:24:29Z

version 4.19.4 on python 3.6.3

casperdcl · 2017-11-19T14:18:33Z

Sounds like disabled locks or non-functioning threading. Are you using windows?

You could disable the monitor thread by doing

from tqdm import tqdm
tqdm.monitor_interval = 0

before the rest of your code

solomatov · 2017-11-19T14:45:46Z

I have the same issue on OSX

utkonos · 2017-11-20T04:05:32Z

I am on macOS as well.

drnextgis · 2017-12-01T16:17:09Z

Faced with the same issue on Ubuntu (tqdm==4.19.4, Python 2.7.12). Workaround by setting monitor_interval works.

Exception in thread Thread-2:                                                                                                        | 9735/193414 [00:05<01:40, 1822.75it/s]
Traceback (most recent call last):
  File "/usr/lib/python2.7/threading.py", line 801, in __bootstrap_inner
    self.run()
  File "/home/rykov/sandbox/ngdownload/env/local/lib/python2.7/site-packages/tqdm/_tqdm.py", line 144, in run
    for instance in self.tqdm_cls._instances:
  File "/home/rykov/sandbox/ngdownload/env/lib/python2.7/_weakrefset.py", line 60, in __iter__
    for itemref in self.data:
RuntimeError: Set changed size during iteration

utkonos · 2017-12-26T03:48:41Z

I'm still having this same exception occur with the current version, 4.19.5.

I also can confirm that tqdm.monitor_interval = 0 is a valid workaround.

wilrich-msft · 2018-01-02T21:05:53Z

I have the same issue on Win10 / 3.6.3 |Anaconda custom (64-bit)| (default, Nov 8 2017, 15:10:56) [MSC v.1900 64 bit (AMD64)]:

from sklearn.cross_validation import KFold
scores = []
N_FOLDS = 10
from sklearn.utils import shuffle
X, Y = shuffle(X, Y, random_state=0)
cv = KFold(n=len(X), n_folds=N_FOLDS)

for train, test in tqdm(cv, total=N_FOLDS):    
    clf = neighbors.KNeighborsClassifier()
    clf.fit(X[train], Y[train])
    scores.append(clf.score(X[test], Y[test]))

resulting in

Exception in thread Thread-33:
Traceback (most recent call last):
  File "C:\local\Anaconda3-4.1.1-Windows-x86_64\lib\threading.py", line 916, in _bootstrap_inner
    self.run()
  File "C:\local\Anaconda3-4.1.1-Windows-x86_64\lib\site-packages\tqdm\_tqdm.py", line 144, in run
    for instance in self.tqdm_cls._instances:
  File "C:\local\Anaconda3-4.1.1-Windows-x86_64\lib\_weakrefset.py", line 60, in __iter__
    for itemref in self.data:
RuntimeError: Set changed size during iteration

MattKleinsmith · 2018-01-21T08:08:16Z

I received this error after switching to starmap from map, from the multiprocessing package. I went back to map and the error went away.

Specs: Ubuntu 16.04, nvidia docker, jupyter notebook 5.2.2, python 3.6.4, anaconda, tqdm 4.19.4, tqdm_notebook

Starmap code:

from multiprocessing import Pool

def get_and_save_patch(out_path, img, i):
    img = random_crop(img, PATCH_SIZE)
    out_path_png = out_path.split('.')[0] + "_" + str(i) + "_.png"
    cv2.imwrite(out_path_png, img)

cameras = sorted(os.listdir(TRN_DIR))
for camera in tqdm_notebook(cameras):
    in_dir = osp.join(TRN_DIR, camera)
    out_dir = osp.join(TRN_64_DIR, camera)
    os.makedirs(out_dir)
    in_filenames = sort_files_by_int(os.listdir(in_dir))
    for in_filename in tqdm_notebook(in_filenames):
        in_path = osp.join(in_dir, in_filename)
        out_path = osp.join(out_dir, in_filename)
        img = np.array(cv2.imread(in_path))
        args = [(out_path, img, i) for i in range(NUM_PATCHES_PER_IMG)]  # Shallow copies.
        with Pool(NPROC) as pool:  # Multiprocessing.
            pool.starmap(get_and_save_patch, args)

Map code:

from multiprocessing import Pool

def get_and_save_patch(args):  ######### Diff.
    out_path, img, i = args  ######### Diff.
    img = random_crop(img, PATCH_SIZE)
    out_path_png = out_path.split('.')[0] + "_" + str(i) + "_.png"
    cv2.imwrite(out_path_png, img)

cameras = sorted(os.listdir(TRN_DIR))
for camera in tqdm_notebook(cameras):
    in_dir = osp.join(TRN_DIR, camera)
    out_dir = osp.join(TRN_64_DIR, camera)
    os.makedirs(out_dir)
    in_filenames = sort_files_by_int(os.listdir(in_dir))
    for in_filename in tqdm_notebook(in_filenames):
        in_path = osp.join(in_dir, in_filename)
        out_path = osp.join(out_dir, in_filename)
        img = np.array(cv2.imread(in_path))
        args = [(out_path, img, i) for i in range(NUM_PATCHES_PER_IMG)]  # Shallow copies.
        with Pool(NPROC) as pool:  # Multiprocessing.
            pool.map(get_and_save_patch, args)  ######### Diff.

utkonos · 2018-01-21T17:16:41Z

Without digging even deeper, I think the comment 'Shallow copies.' is telling. I have encountered exceptions just like this if I iterate on a shallow copy of an interable and operate on that same iterable inside the loop using del or pop or such operations. I bet that somewhere in there is copy.copy() rather than copy.deepcopy(). However, this is just a guess.

utkonos · 2018-01-21T17:18:51Z

Also, unfortunately, the workaround found above does not work 100% of the time. I am still encountering this bug even with the workaround tqdm.monitor_interval = 0 in place.

Add suggested workaround for tqdm/tqdm#481

wjaskowski · 2018-03-22T10:27:50Z

I get it on python 3.6.4 and tqdm 4.11.2 on Ubuntu 16.04 using jupyter notebook

utkonos · 2018-03-24T20:00:19Z

This is still happening for me on tqdm (4.19.8) on macOS using jupyter notebook.

exception in thread Thread-124:
Traceback (most recent call last):
  File "/usr/local/Cellar/python/3.6.4_4/Frameworks/Python.framework/Versions/3.6/lib/python3.6/threading.py", line 916, in _bootstrap_inner
    self.run()
  File "/usr/local/lib/python3.6/site-packages/tqdm/_monitor.py", line 63, in run
    for instance in self.tqdm_cls._instances:
  File "/usr/local/Cellar/python/3.6.4_4/Frameworks/Python.framework/Versions/3.6/lib/python3.6/_weakrefset.py", line 60, in __iter__
    for itemref in self.data:
RuntimeError: Set changed size during iteration

habibutsu · 2018-03-28T14:02:24Z

I am found old fix, but which is absent in current code
64f5e73

casperdcl · 2018-03-28T23:18:17Z

that was a hack before we introduced locks, which should have eliminated the problem. It would be nice to figure out and tackle the issue rather than reintroducing the hack

habibutsu · 2018-03-29T06:49:39Z

@casperdcl I think this is bad approach. This sounds for me like that you suggest do not reanimate a patient until understand what is reason of incident or don't save the somebody from fire until not understand of fire reason.

A am agree that needed to figure out how to fix this correctly, but in the same time until this is not clear would be nice to make this hack.

- suppress `RuntimeError: Set changed size during iteration` (#481) - partially re-add 64f5e73 Happy Easter!

casperdcl · 2018-04-02T22:30:47Z

xD not sure whether the analogy transfers, but re-added this all the same in devel

javadnoorb · 2018-04-04T14:13:19Z

Same warning in Ubuntu 16.04.4 with tqdm4.19.9 and Python3.6.3

casperdcl · 2018-04-05T16:33:56Z

@javadnoorb if you upgrade to v4.20.0, do you still get this?

javadnoorb · 2018-04-05T18:07:53Z

@casperdcl I'm not sure. I can't get the upgrade with conda. But the error is not easily reproducible. I'm not getting it again with v4.19.9

casperdcl · 2018-04-05T19:37:07Z

v4.20.0 should be available: https://anaconda.org/conda-forge/tqdm

TinaMurimi · 2018-05-04T06:39:44Z

The latest version of tqdm 4.23.3 has a warning TqdmSynchronisationWarning. This makes my download fail. Downgraded the tqdm version to 4.19.7 which works.
OS: Mac
Python version: 3.6.5

casperdcl · 2018-05-04T13:38:07Z

Warnings should not cause failures. They print information. The correct way to suppress this is:

from tqdm import tqdm, TqdmSynchronisationWarning
import warnings

with warnings.catch_warnings():
    warnings.simplefilter("ignore", TqdmSynchronisationWarning)
    # some code here...

RTM:
https://docs.python.org/2/library/warnings.html#temporarily-suppressing-warnings
https://docs.python.org/3.6/library/warnings.html#temporarily-suppressing-warnings

This issue is patched in tqdm>=4.20.0, where the Error is replaced with a Warning.

dhoomakethu · 2018-07-03T03:40:58Z

So I was encountering random RuntimeError: Set changed size during iteration with tqdm 4.19.5 and came here to find a solution, I upgraded to latest (4.23.4) and now I get another error Exception KeyError: KeyError(<weakref at 0x7f47a5664cb0; to 'tqdm' at 0x7f47a60be9d0>,) in <bound method tqdm.__del__.

--Update--
it turns out that I was also using tqdm.monitor_interval=0 along with catching the warnings . The warning is printed when we break out of tqdm loop . Here is an example to reproduce (tqdm v4.23.4)

import requests
import time
from tqdm import tqdm, TqdmSynchronisationWarning
import warnings
from requests.exceptions import ConnectionError
max_tries = 100
sentinel_addr = "http://localhost:9999/api/action"
payload = {"action": "sign"}
timeout = 15

with warnings.catch_warnings():
    warnings.simplefilter("ignore", TqdmSynchronisationWarning)
    for tries in tqdm(range(max_tries), ascii=True, leave=False,
                      desc="Waiting for Server '{}'"
                           " (In Seconds)".format(sentinel_addr),
                      unit="seconds",
                      bar_format='{desc}: {elapsed}'):
        try:
            resp = requests.post(sentinel_addr, json=payload,
                                 timeout=timeout)
            if resp.status_code == 200:
                break
            if tries == 10:
                # Lets break out of the loop 
                break
        except ConnectionError:
            if tries == 10:
                # Lets break out of the loop 
                break
        time.sleep(1)

BearTian · 2018-10-13T03:04:34Z

Exception in thread Thread-5:
Traceback (most recent call last):
File "C:\Users\max\AppData\Local\Programs\Python\Python37\lib\threading.py", line 917, in _bootstrap_inner
self.run()
File "D:\Proj\antwork\code\antwork_backend\server\data_handlers\order\playback_hd.py", line 147, in run
send_to_ws(channel=self.room_name, data=json.loads(msg))
File "D:\Proj\antwork\code\antwork_backend\server\data_exchange.py", line 26, in send_to_ws
asyncio.run(channel_layer.group_send(channel, send_data))
File "C:\Users\max\AppData\Local\Programs\Python\Python37\lib\asyncio\runners.py", line 46, in run
_cancel_all_tasks(loop)
File "C:\Users\max\AppData\Local\Programs\Python\Python37\lib\asyncio\runners.py", line 54, in _cancel_all_tasks
to_cancel = tasks.all_tasks(loop)
File "C:\Users\max\AppData\Local\Programs\Python\Python37\lib\asyncio\tasks.py", line 38, in all_tasks
return {t for t in _all_tasks
File "C:\Users\max\AppData\Local\Programs\Python\Python37\lib\asyncio\tasks.py", line 38, in
return {t for t in _all_tasks
File "C:\Users\max\AppData\Local\Programs\Python\Python37\lib_weakrefset.py", line 61, in iter
for itemref in self.data:
RuntimeError: Set changed size during iteration

casperdcl · 2019-06-18T18:59:52Z

should be fixed now (tqdm>=4.32.2)

closed by #700 & #754

mondeja · 2021-01-17T12:22:16Z

I'm getting the same error in Python3.6 with tqdm v4.56.0. This does not happen in Python 3.7, 3.8 neither 3.9.

afonsoalbrecht · 2021-01-24T18:01:33Z

should be fixed now (tqdm>=4.32.2)

closed by #700 & #754

no, this isn't fixed. I've got this problem now.

afonsoalbrecht · 2021-01-24T21:06:29Z

is there any proper solution to this problem instead of this workaround?

Dobatymo · 2021-08-18T09:27:44Z

I am still getting the error with 4.61.1 on windows.
The problem only occurs sometimes and is related to tqdm being called from different threads.

casperdcl · 2021-08-22T19:27:47Z

@Dobatymo can you post a traceback?

Dobatymo · 2021-08-22T23:26:56Z

I forgot which tqdm version caused this specific traceback, but it's a recent one and I think the traceback was the same for the latest version too.

Exception in thread Thread-5:
Traceback (most recent call last):
  File "C:\Program Files\Python37\lib\threading.py", line 926, in _bootstrap_inner
    self.run()
  File "C:\Program Files\Python37\lib\site-packages\tqdm\_monitor.py", line 78, in run
    instances = self.get_instances()
  File "C:\Program Files\Python37\lib\site-packages\tqdm\_monitor.py", line 58, in get_instances
    return [i for i in self.tqdm_cls._instances.copy()
  File "C:\Program Files\Python37\lib\_weakrefset.py", line 92, in copy
    return self.__class__(self)
  File "C:\Program Files\Python37\lib\_weakrefset.py", line 50, in __init__
    self.update(data)
  File "C:\Program Files\Python37\lib\_weakrefset.py", line 119, in update
    for element in other:
  File "C:\Program Files\Python37\lib\_weakrefset.py", line 60, in __iter__
    for itemref in self.data:
RuntimeError: Set changed size during iteration

So the copy() actually doesn't fix anything, as the copy itself requires iterating...

Sorry I don't have time to fully debug this, but I get this error commonly with internetarchive which uses urllib3. In this case the tqdm object is passed wrapping a request to urllib3. urllib3 uses a threadpool for it's request. So I think the tqdm object is created in one thread, and then iterated in another (the urllib3 threadpool).

- fixes #481

vTuanpham · 2024-09-12T04:23:16Z

Still the same error

5879.1s	371	Traceback (most recent call last):
5879.1s	372	  File "/opt/conda/lib/python3.10/threading.py", line 1016, in _bootstrap_inner
5879.1s	373	    self.run()
5879.1s	374	  File "/opt/conda/lib/python3.10/site-packages/tqdm/_monitor.py", line 69, in run
5879.1s	375	    instances = self.get_instances()
5879.1s	376	  File "/opt/conda/lib/python3.10/site-packages/tqdm/_monitor.py", line 49, in get_instances
5879.1s	377	    return [i for i in self.tqdm_cls._instances.copy()
5879.1s	378	  File "/opt/conda/lib/python3.10/_weakrefset.py", line 97, in copy
5879.1s	379	    return self.__class__(self)
5879.1s	380	  File "/opt/conda/lib/python3.10/_weakrefset.py", line 51, in __init__
5879.1s	381	    self.update(data)
5879.1s	382	  File "/opt/conda/lib/python3.10/_weakrefset.py", line 124, in update
5879.1s	383	    for element in other:
5879.1s	384	  File "/opt/conda/lib/python3.10/_weakrefset.py", line 65, in __iter__
5879.1s	385	    for itemref in self.data:
5879.1s	386	RuntimeError: Set changed size during iteration

tqdm==4.66.1
Notebook

Spawn about 66 separate progress bars, the error seem to happen randomly.

casperdcl mentioned this issue Dec 27, 2017

tqdm issues with the debugger ipdb #492

Open

omelnikov mentioned this issue Dec 28, 2017

RuntimeError: Set changed size during iteration pymc-devs/pymc#2670

Closed

danielhers added a commit to danielhers/tupa that referenced this issue Jan 18, 2018

Workaround for tqdm/tqdm#481

0148777

casperdcl assigned lrq3000 Jan 21, 2018

casperdcl added p0-bug-critical ☢ Exception rasing help wanted 🙏 We need you (discussion or implementation) labels Jan 21, 2018

timodonnell added a commit to openvax/mhcflurry that referenced this issue Feb 8, 2018

Workaround tqdm issue

877c60a

Add suggested workaround for tqdm/tqdm#481

starfys mentioned this issue Mar 6, 2018

Space between lines keeps increasing bstriner/keras-tqdm#21

Open

bartolsthoorn mentioned this issue Mar 18, 2018

Freeze of GlobalSimilarity.py lab-cosmo/glosim2#1

Closed

casperdcl added a commit that referenced this issue Apr 2, 2018

TqdmSynchronisationWarning

0e8f839

- suppress `RuntimeError: Set changed size during iteration` (#481) - partially re-add 64f5e73 Happy Easter!

casperdcl closed this as completed May 4, 2018

casperdcl mentioned this issue May 4, 2018

exception and thread safety #548

Open

6 tasks

guozhongluo mentioned this issue May 7, 2018

A problem in running run.sh MVIG-SJTU/AlphaPose#58

Closed

TanMan1217 mentioned this issue May 8, 2018

Using the -e switch in Windows - Question slhck/ffmpeg-normalize#70

Closed

yarikoptic mentioned this issue May 15, 2018

TqdmSynchronisationWarning: Set changed size during iteration (see https://github.com/tqdm/tqdm/issues/481) datalad/datalad#2468

Closed

ungarj added a commit to ungarj/mapchete that referenced this issue May 31, 2018

workaround for tqdm issue tqdm/tqdm#481

e47d8a1

nobicycle mentioned this issue Jul 9, 2018

Version 0.10 Server SSL Errors learningequality/kolibri#3963

Closed

neroransom mentioned this issue Mar 16, 2019

爬虫过半终止，未能生成json文件 zengyu714/weibot#1

Open

vnetserg mentioned this issue Mar 21, 2019

Fix 'Set changed size during iteration' bug #700

Closed

casperdcl reopened this Mar 22, 2019

casperdcl closed this as completed Jun 18, 2019

DASSR mentioned this issue Sep 13, 2019

Coursera Deep Learning ZEMUSHKA/coursera-aml-docker#6

Open

draconidsmxz mentioned this issue Jul 16, 2020

multi-thread problem MVIG-SJTU/AlphAction#8

Closed

mondeja mentioned this issue Jan 17, 2021

Add AudioClip stereo example Zulko/moviepy#1459

Merged

casperdcl reopened this Jan 26, 2021

kayhayen mentioned this issue Mar 26, 2021

Error with tqdm during compilation (.... size changed during iteration) Nuitka/Nuitka#1040

Closed

casperdcl added a commit that referenced this issue Aug 23, 2021

fix monitor RuntimeError

b6326c2

- fixes #481

utkonos closed this as completed Aug 18, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

RuntimeError: Set changed size during iteration #481

RuntimeError: Set changed size during iteration #481

utkonos commented Nov 19, 2017 •

edited by casperdcl

Loading

utkonos commented Nov 19, 2017

casperdcl commented Nov 19, 2017

solomatov commented Nov 19, 2017

utkonos commented Nov 20, 2017

drnextgis commented Dec 1, 2017 •

edited

Loading

utkonos commented Dec 26, 2017

wilrich-msft commented Jan 2, 2018

MattKleinsmith commented Jan 21, 2018 •

edited

Loading

utkonos commented Jan 21, 2018

utkonos commented Jan 21, 2018

wjaskowski commented Mar 22, 2018 •

edited

Loading

utkonos commented Mar 24, 2018 •

edited

Loading

habibutsu commented Mar 28, 2018

casperdcl commented Mar 28, 2018

habibutsu commented Mar 29, 2018 •

edited

Loading

casperdcl commented Apr 2, 2018

javadnoorb commented Apr 4, 2018

casperdcl commented Apr 5, 2018

javadnoorb commented Apr 5, 2018

casperdcl commented Apr 5, 2018

TinaMurimi commented May 4, 2018

casperdcl commented May 4, 2018

dhoomakethu commented Jul 3, 2018 •

edited

Loading

BearTian commented Oct 13, 2018

casperdcl commented Jun 18, 2019

mondeja commented Jan 17, 2021 •

edited

Loading

afonsoalbrecht commented Jan 24, 2021

afonsoalbrecht commented Jan 24, 2021

Dobatymo commented Aug 18, 2021

casperdcl commented Aug 22, 2021

Dobatymo commented Aug 22, 2021 •

edited

Loading

vTuanpham commented Sep 12, 2024

RuntimeError: Set changed size during iteration #481

RuntimeError: Set changed size during iteration #481

Comments

utkonos commented Nov 19, 2017 • edited by casperdcl Loading

utkonos commented Nov 19, 2017

casperdcl commented Nov 19, 2017

solomatov commented Nov 19, 2017

utkonos commented Nov 20, 2017

drnextgis commented Dec 1, 2017 • edited Loading

utkonos commented Dec 26, 2017

wilrich-msft commented Jan 2, 2018

MattKleinsmith commented Jan 21, 2018 • edited Loading

utkonos commented Jan 21, 2018

utkonos commented Jan 21, 2018

wjaskowski commented Mar 22, 2018 • edited Loading

utkonos commented Mar 24, 2018 • edited Loading

habibutsu commented Mar 28, 2018

casperdcl commented Mar 28, 2018

habibutsu commented Mar 29, 2018 • edited Loading

casperdcl commented Apr 2, 2018

javadnoorb commented Apr 4, 2018

casperdcl commented Apr 5, 2018

javadnoorb commented Apr 5, 2018

casperdcl commented Apr 5, 2018

TinaMurimi commented May 4, 2018

casperdcl commented May 4, 2018

dhoomakethu commented Jul 3, 2018 • edited Loading

BearTian commented Oct 13, 2018

casperdcl commented Jun 18, 2019

mondeja commented Jan 17, 2021 • edited Loading

afonsoalbrecht commented Jan 24, 2021

afonsoalbrecht commented Jan 24, 2021

Dobatymo commented Aug 18, 2021

casperdcl commented Aug 22, 2021

Dobatymo commented Aug 22, 2021 • edited Loading

vTuanpham commented Sep 12, 2024

utkonos commented Nov 19, 2017 •

edited by casperdcl

Loading

drnextgis commented Dec 1, 2017 •

edited

Loading

MattKleinsmith commented Jan 21, 2018 •

edited

Loading

wjaskowski commented Mar 22, 2018 •

edited

Loading

utkonos commented Mar 24, 2018 •

edited

Loading

habibutsu commented Mar 29, 2018 •

edited

Loading

dhoomakethu commented Jul 3, 2018 •

edited

Loading

mondeja commented Jan 17, 2021 •

edited

Loading

Dobatymo commented Aug 22, 2021 •

edited

Loading