Periodic bandwidth usage when using TensorBoard in Colab

[shashikdm]

Environment information

Diagnostics

Diagnostics output

--- check: autoidentify
INFO: diagnose_tensorboard.py version d515ab103e2b1cfcea2b096187741a0eeb8822ef

--- check: general
INFO: sys.version_info: sys.version_info(major=3, minor=6, micro=9, releaselevel='final', serial=0)
INFO: os.name: posix
INFO: os.uname(): posix.uname_result(sysname='Linux', nodename='da75533f442f', release='4.14.137+', version='#1 SMP Thu Aug 8 02:47:02 PDT 2019', machine='x86_64')
INFO: sys.getwindowsversion(): N/A

--- check: package_management
INFO: has conda-meta: False
INFO: $VIRTUAL_ENV: None

--- check: installed_packages
INFO: installed: tensorboard==1.15.0
INFO: installed: tensorflow==1.15.0
INFO: installed: tensorflow-estimator==1.15.1

--- check: tensorboard_python_version
INFO: tensorboard.version.VERSION: '1.15.0'

--- check: tensorflow_python_version
INFO: tensorflow.__version__: '1.15.0'
INFO: tensorflow.__git_version__: 'v1.15.0-0-g590d6eef7e'

--- check: tensorboard_binary_path
INFO: which tensorboard: b'/usr/local/bin/tensorboard\n'

--- check: addrinfos
socket.has_ipv6 = True
socket.AF_UNSPEC = <AddressFamily.AF_UNSPEC: 0>
socket.SOCK_STREAM = <SocketKind.SOCK_STREAM: 1>
socket.AI_ADDRCONFIG = <AddressInfo.AI_ADDRCONFIG: 32>
socket.AI_PASSIVE = <AddressInfo.AI_PASSIVE: 1>
Loopback flags: <AddressInfo.AI_ADDRCONFIG: 32>
Loopback infos: [(<AddressFamily.AF_INET: 2>, <SocketKind.SOCK_STREAM: 1>, 6, '', ('127.0.0.1', 0))]
Wildcard flags: <AddressInfo.AI_PASSIVE: 1>
Wildcard infos: [(<AddressFamily.AF_INET: 2>, <SocketKind.SOCK_STREAM: 1>, 6, '', ('0.0.0.0', 0)), (<AddressFamily.AF_INET6: 10>, <SocketKind.SOCK_STREAM: 1>, 6, '', ('::', 0, 0, 0))]

--- check: readable_fqdn
INFO: socket.getfqdn(): 'da75533f442f'

--- check: stat_tensorboardinfo
INFO: directory: /tmp/.tensorboard-info
INFO: .tensorboard-info directory does not exist

--- check: source_trees_without_genfiles
INFO: tensorboard_roots (1): ['/usr/local/lib/python3.6/dist-packages']; bad_roots (0): []

--- check: full_pip_freeze
INFO: pip freeze --all:
absl-py==0.9.0
alabaster==0.7.12
albumentations==0.1.12
altair==4.0.0
asgiref==3.2.3
astor==0.8.1
astropy==4.0
atari-py==0.2.6
atomicwrites==1.3.0
attrs==19.3.0
audioread==2.1.8
autograd==1.3
Babel==2.8.0
backcall==0.1.0
backports.tempfile==1.0
backports.weakref==1.0.post1
beautifulsoup4==4.6.3
bleach==3.1.0
blis==0.2.4
bokeh==1.4.0
boto==2.49.0
boto3==1.10.47
botocore==1.13.47
Bottleneck==1.3.1
branca==0.3.1
bs4==0.0.1
bz2file==0.98
cachetools==4.0.0
certifi==2019.11.28
cffi==1.13.2
chainer==6.5.0
chardet==3.0.4
chart-studio==1.0.0
Click==7.0
cloudpickle==1.2.2
cmake==3.12.0
colorlover==0.3.0
community==1.0.0b1
contextlib2==0.5.5
convertdate==2.2.0
coverage==3.7.1
coveralls==0.5
crcmod==1.7
cufflinks==0.17.0
cupy-cuda101==6.5.0
cvxopt==1.2.3
cvxpy==1.0.25
cycler==0.10.0
cymem==2.0.3
Cython==0.29.14
daft==0.0.4
dask==2.9.1
dataclasses==0.7
datascience==0.10.6
decorator==4.4.1
defusedxml==0.6.0
descartes==1.1.0
dill==0.3.1.1
distributed==1.25.3
Django==3.0.2
dlib==19.18.0
dm-sonnet==1.35
docopt==0.6.2
docutils==0.15.2
dopamine-rl==1.0.5
earthengine-api==0.1.209
easydict==1.9
ecos==2.0.7.post1
editdistance==0.5.3
en-core-web-sm==2.1.0
entrypoints==0.3
et-xmlfile==1.0.1
fa2==0.3.5
fancyimpute==0.4.3
fastai==1.0.60
fastcache==1.1.0
fastdtw==0.3.4
fastprogress==0.2.2
fastrlock==0.4
fbprophet==0.5
feather-format==0.4.0
featuretools==0.4.1
filelock==3.0.12
fix-yahoo-finance==0.0.22
Flask==1.1.1
folium==0.8.3
fsspec==0.6.2
future==0.16.0
gast==0.2.2
GDAL==2.2.2
gdown==3.6.4
gensim==3.6.0
geographiclib==1.50
geopy==1.17.0
gevent==1.4.0
gin-config==0.3.0
glob2==0.7
google==2.0.3
google-api-core==1.15.0
google-api-python-client==1.7.11
google-auth==1.4.2
google-auth-httplib2==0.0.3
google-auth-oauthlib==0.4.1
google-cloud-bigquery==1.21.0
google-cloud-core==1.0.3
google-cloud-datastore==1.8.0
google-cloud-language==1.2.0
google-cloud-storage==1.16.2
google-cloud-translate==1.5.0
google-colab==1.0.0
google-pasta==0.1.8
google-resumable-media==0.4.1
googleapis-common-protos==1.6.0
googledrivedownloader==0.4
graph-nets==1.0.5
graphviz==0.10.1
greenlet==0.4.15
grpcio==1.15.0
gspread==3.0.1
gspread-dataframe==3.0.3
gunicorn==20.0.4
gym==0.15.4
h5py==2.8.0
HeapDict==1.0.1
holidays==0.9.12
html5lib==1.0.1
httpimport==0.5.18
httplib2==0.11.3
humanize==0.5.1
hyperopt==0.1.2
ideep4py==2.0.0.post3
idna==2.8
image==1.5.27
imageio==2.4.1
imagesize==1.2.0
imbalanced-learn==0.4.3
imblearn==0.0
imgaug==0.2.9
importlib-metadata==1.4.0
imutils==0.5.3
inflect==2.1.0
intel-openmp==2020.0.133
intervaltree==2.1.0
ipykernel==4.6.1
ipython==5.5.0
ipython-genutils==0.2.0
ipython-sql==0.3.9
ipywidgets==7.5.1
itsdangerous==1.1.0
jax==0.1.52
jaxlib==0.1.36
jdcal==1.4.1
jedi==0.15.2
jieba==0.40
Jinja2==2.10.3
jmespath==0.9.4
joblib==0.14.1
jpeg4py==0.1.4
jsonschema==2.6.0
jupyter==1.0.0
jupyter-client==5.3.4
jupyter-console==5.2.0
jupyter-core==4.6.1
kaggle==1.5.6
kapre==0.1.3.1
Keras==2.2.5
Keras-Applications==1.0.8
Keras-Preprocessing==1.1.0
keras-vis==0.4.1
kfac==0.2.0
kiwisolver==1.1.0
knnimpute==0.1.0
librosa==0.6.3
lightgbm==2.2.3
llvmlite==0.31.0
lmdb==0.98
lucid==0.3.8
lunardate==0.2.0
lxml==4.2.6
magenta==0.3.19
Markdown==3.1.1
MarkupSafe==1.1.1
matplotlib==3.1.2
matplotlib-venn==0.11.5
mesh-tensorflow==0.1.8
mido==1.2.6
mir-eval==0.5
missingno==0.4.2
mistune==0.8.4
mizani==0.6.0
mkl==2019.0
mlxtend==0.14.0
more-itertools==8.0.2
moviepy==0.2.3.5
mpi4py==3.0.3
mpmath==1.1.0
msgpack==0.5.6
multiprocess==0.70.9
multitasking==0.0.9
murmurhash==1.0.2
music21==5.5.0
natsort==5.5.0
nbconvert==5.6.1
nbformat==5.0.3
networkx==2.4
nibabel==2.3.3
nltk==3.2.5
notebook==5.2.2
np-utils==0.5.12.1
numba==0.47.0
numexpr==2.7.1
numpy==1.17.5
nvidia-ml-py3==7.352.0
oauth2client==4.1.3
oauthlib==3.1.0
okgrade==0.4.3
opencv-contrib-python==4.1.2.30
opencv-python==4.1.2.30
openpyxl==2.5.9
opt-einsum==3.1.0
osqp==0.6.1
packaging==20.0
palettable==3.3.0
pandas==0.25.3
pandas-datareader==0.7.4
pandas-gbq==0.11.0
pandas-profiling==1.4.1
pandocfilters==1.4.2
parso==0.5.2
pathlib==1.0.1
patsy==0.5.1
pexpect==4.7.0
pickleshare==0.7.5
Pillow==6.2.2
pip==19.3.1
pip-tools==4.2.0
plac==0.9.6
plotly==4.4.1
plotnine==0.6.0
pluggy==0.7.1
portpicker==1.2.0
prefetch-generator==1.0.1
preshed==2.0.1
pretty-midi==0.2.8
prettytable==0.7.2
progressbar2==3.38.0
prometheus-client==0.7.1
promise==2.3
prompt-toolkit==1.0.18
protobuf==3.10.0
psutil==5.4.8
psycopg2==2.7.6.1
ptyprocess==0.6.0
py==1.8.1
pyarrow==0.14.1
pyasn1==0.4.8
pyasn1-modules==0.2.7
pycocotools==2.0.0
pycparser==2.19
pydata-google-auth==0.2.1
pydot==1.3.0
pydot-ng==2.0.0
pydotplus==2.0.2
PyDrive==1.3.1
pyemd==0.5.1
pyglet==1.3.2
Pygments==2.1.3
pygobject==3.26.1
pymc3==3.7
PyMeeus==0.3.6
pymongo==3.10.0
pymystem3==0.2.0
PyOpenGL==3.1.5
pyparsing==2.4.6
pypng==0.0.20
pyrsistent==0.15.7
pysndfile==1.3.8
PySocks==1.7.1
pystan==2.19.1.1
pytest==3.6.4
python-apt==1.6.4
python-chess==0.23.11
python-dateutil==2.6.1
python-louvain==0.13
python-rtmidi==1.3.1
python-slugify==4.0.0
python-utils==2.3.0
pytz==2018.9
PyWavelets==1.1.1
PyYAML==3.13
pyzmq==17.0.0
qtconsole==4.6.0
regex==2019.12.20
requests==2.21.0
requests-oauthlib==1.3.0
resampy==0.2.2
retrying==1.3.3
rpy2==2.9.5
rsa==4.0
s3fs==0.4.0
s3transfer==0.2.1
scikit-image==0.16.2
scikit-learn==0.22.1
scipy==1.4.1
screen-resolution-extra==0.0.0
scs==2.1.1.post2
seaborn==0.9.0
semantic-version==2.8.4
Send2Trash==1.5.0
setuptools==42.0.2
setuptools-git==1.2
Shapely==1.6.4.post2
simplegeneric==0.8.1
six==1.12.0
sklearn==0.0
sklearn-pandas==1.8.0
smart-open==1.9.0
snowballstemmer==2.0.0
sortedcontainers==2.1.0
spacy==2.1.9
Sphinx==1.8.5
sphinxcontrib-websupport==1.1.2
SQLAlchemy==1.3.12
sqlparse==0.3.0
srsly==1.0.1
stable-baselines==2.2.1
statsmodels==0.10.2
sympy==1.1.1
tables==3.4.4
tabulate==0.8.6
tblib==1.6.0
tensor2tensor==1.14.1
tensorboard==1.15.0
tensorboardcolab==0.0.22
tensorflow==1.15.0
tensorflow-datasets==1.3.2
tensorflow-estimator==1.15.1
tensorflow-gan==2.0.0
tensorflow-hub==0.7.0
tensorflow-metadata==0.15.2
tensorflow-privacy==0.2.2
tensorflow-probability==0.7.0
termcolor==1.1.0
terminado==0.8.3
testpath==0.4.4
text-unidecode==1.3
textblob==0.15.3
textgenrnn==1.4.1
tflearn==0.3.2
Theano==1.0.4
thinc==7.0.8
toolz==0.10.0
torch==1.3.1
torchsummary==1.5.1
torchtext==0.3.1
torchvision==0.4.2
tornado==4.5.3
tqdm==4.28.1
traitlets==4.3.3
tweepy==3.6.0
typing==3.6.6
typing-extensions==3.6.6
tzlocal==1.5.1
umap-learn==0.3.10
uritemplate==3.0.1
urllib3==1.24.3
vega-datasets==0.8.0
wasabi==0.6.0
wcwidth==0.1.8
webencodings==0.5.1
Werkzeug==0.16.0
wheel==0.33.6
widgetsnbextension==3.5.1
wordcloud==1.5.0
wrapt==1.11.2
xarray==0.14.1
xgboost==0.90
xkit==0.0.0
xlrd==1.1.0
xlwt==1.3.0
yellowbrick==0.9.1
zict==1.0.0
zipp==1.0.0
zmq==0.0.0

Browser type: Mozilla Firefox 72.0.1

Screenshot

Issue description

While using TensorBoard in colab with %tensorboard magics, I noticed a periodic peak in my upload speed. What could TensorBoard possibly need to upload which requires so much bandwidth? Not sure if this is a bug.

[wchargin]

Hi @shashikdm! This is interesting. I suspect that this is an artifact
of Colab’s cell caching. Whenever the output frame (here, TensorBoard)
makes an HTTP request to the backend running in the Colab VM, Colab
saves the content of that request in the notebook itself, so that if you
send someone else the notebook they can still see a working TensorBoard
even without running the server locally. The TensorBoard frontend has a
periodic auto-reload by default, so the network requests due to this
auto-reloading will be cached, and your machine will upload that data to
Colab so that it can be stored in the notebook.

To confirm that this is what’s happening, you could try disabling
auto-reload (click the gear icon in the top right) and seeing if the
bandwidth usage persists. I can reproduce the behavior that you’re
seeing, and turning off auto-reload causes it to disappear on my end.

In case you’re concerned, we (the TensorBoard team) do not have any
analytics hooked up to %tensorboard: the TensorBoard packages
installed on Colab by default just the standard tensorboard packages
from PyPI, which do not contain any tracking.

[shashikdm]

thank you!
I think Colab cell caching this is unavoidable.
I'll do

you could try disabling
auto-reload (click the gear icon in the top right)

from now on to save bandwidth

[blois]

The cell caching can be disabled with the caching parameter, I believe Colab defaults it to off but that tensorboard is enabling it. Perhaps this could be exposed as an option?