Skip to content

Add TensorBoard to Kaggle Kernels#494

Merged
emzeq merged 4 commits intomasterfrom
tensorboard
Feb 28, 2019
Merged

Add TensorBoard to Kaggle Kernels#494
emzeq merged 4 commits intomasterfrom
tensorboard

Conversation

@emzeq
Copy link
Contributor

@emzeq emzeq commented Feb 26, 2019

To use:

%load_ext tensorboard.notebook
%tensorboard --logdir /logs

@emzeq emzeq requested review from corrieann and sebbov February 26, 2019 05:32
sebbov
sebbov reviewed Feb 26, 2019
View reviewed changes
Copy link
Contributor

@sebbov sebbov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

General requests:

  • Have you tested it end-to-end in Kaggle Kernels? For both CPU and GPU? Can you include the steps and results in the PR description please.
  • Can you include a test under tests/ that exercises this and perhaps any existing Jupyter functionality this might affect?
  • Please make sure our internal worker integration tests passes with the update to this image.

@sebbov sebbov requested review from rosbo and sebbov February 26, 2019 16:53
sebbov
sebbov reviewed Feb 26, 2019
View reviewed changes
Copy link
Contributor

@sebbov sebbov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added @rosbo since he's the primary gatekeeper here.

@rosbo
Copy link
Contributor

rosbo commented Feb 26, 2019

+1 to all of Seb's questions

@emzeq
Copy link
Contributor Author

emzeq commented Feb 27, 2019

Test output (including new test):

/opt/conda/lib/python3.6/site-packages/distributed/utils.py:134: RuntimeWarning: Couldn't detect a suitable IP address for reaching '8.8.8.8', defaulting to '127.0.0.1': [Errno 101] Network is unreachable
  % (host, default, e), RuntimeWarning)
�[0;32m[   INFO   ] �[0mMusicExtractorSVM: no classifier models were configured by default
/opt/conda/lib/python3.6/site-packages/scipy/sparse/sparsetools.py:21: DeprecationWarning: `scipy.sparse.sparsetools` is deprecated!
scipy.sparse.sparsetools is a private module for scipy.sparse, and should not be used.
  _deprecated()
INFO:gensim.summarization.textcleaner:'pattern' package not found; tag filters are not available for English
/opt/conda/lib/python3.6/site-packages/tabulate-0.8.3-py3.6.egg/tabulate.py:192: DeprecationWarning: invalid escape sequence \h
/opt/conda/lib/python3.6/site-packages/tabulate-0.8.3-py3.6.egg/tabulate.py:415: DeprecationWarning: invalid escape sequence \[
/opt/conda/lib/python3.6/site-packages/tabulate-0.8.3-py3.6.egg/tabulate.py:1258: DeprecationWarning: invalid escape sequence \e
/opt/conda/lib/python3.6/site-packages/tabulate-0.8.3-py3.6.egg/tabulate.py:192: DeprecationWarning: invalid escape sequence \h
/opt/conda/lib/python3.6/site-packages/tabulate-0.8.3-py3.6.egg/tabulate.py:415: DeprecationWarning: invalid escape sequence \[
/opt/conda/lib/python3.6/site-packages/tabulate-0.8.3-py3.6.egg/tabulate.py:1258: DeprecationWarning: invalid escape sequence \e
Using TensorFlow backend.
/opt/conda/lib/python3.6/site-packages/tensorflow/python/util/tf_inspect.py:75: DeprecationWarning: inspect.getargspec() is deprecated, use inspect.signature() or inspect.getfullargspec()
  return _inspect.getargspec(target)
WARNING:theano.tensor.blas:Using NumPy C-API based implementation for BLAS functions.
/opt/conda/lib/python3.6/site-packages/theano/tests/main.py:6: DeprecationWarning: Importing from numpy.testing.nosetester is deprecated since 1.15.0, import from numpy.testing instead.
  from numpy.testing.nosetester import NoseTester
INFO:root:Failed to import cuda module: No module named 'caffe2.python.caffe2_pybind11_state_gpu'
INFO:root:Failed to import AMD hip module: No module named 'caffe2.python.caffe2_pybind11_state_hip'
WARNING:root:This caffe2 python run does not have GPU support. Will run in CPU only mode.
/opt/conda/lib/python3.6/site-packages/shap/plots/colors.py:35: DeprecationWarning:

object of type <class 'float'> cannot be safely interpreted as an integer.

/opt/conda/lib/python3.6/site-packages/tensorflow/python/util/tf_inspect.py:75: DeprecationWarning:

inspect.getargspec() is deprecated, use inspect.signature() or inspect.getfullargspec()

test_no_project_with_connected_account (test_bigquery.TestBigQuery) ... INFO:google.auth.compute_engine._metadata:Compute Engine Metadata server unavailable onattempt 1 of 3
INFO:google.auth.compute_engine._metadata:Compute Engine Metadata server unavailable onattempt 2 of 3
INFO:google.auth.compute_engine._metadata:Compute Engine Metadata server unavailable onattempt 3 of 3
ok
test_project_with_connected_account (test_bigquery.TestBigQuery) ... Calling Kaggle.UserSecrets to refresh token.
ok
test_proxy_no_project (test_bigquery.TestBigQuery) ... Using Kaggle's public dataset BigQuery integration.
127.0.0.1 - - [27/Feb/2019 02:33:39] "GET /bigquery/v2/projects/test/datasets HTTP/1.1" 200 -
ok
test_proxy_using_library (test_bigquery.TestBigQuery) ... 127.0.0.1 - - [27/Feb/2019 02:33:40] "GET /bigquery/v2/projects/test/datasets HTTP/1.1" 200 -
ok
test_simultaneous_clients (test_bigquery.TestBigQuery) ... Using Kaggle's public dataset BigQuery integration.
127.0.0.1 - - [27/Feb/2019 02:33:40] "GET /bigquery/v2/projects/test/datasets HTTP/1.1" 200 -
Calling Kaggle.UserSecrets to refresh token.
127.0.0.1 - - [27/Feb/2019 02:33:41] "GET /bigquery/v2/projects/test/datasets HTTP/1.1" 200 -
ok
test_figure (test_bokeh.TestBokeh) ... ok
test_projection (test_cartopy.TestCartopy) ... ok
test_pipeline (test_datashader.TestDatashader) ... ok
test_renderer (test_dipy.TestDipy) ... ok
test_windowing (test_essentia.TestEssentia) ... ok
test_has_version (test_fastai.TestFastAI) ... ok
test_partition (test_fastai.TestFastAI) ... ok
test_tabular (test_fastai.TestFastAI) ... epoch     train_loss  valid_loss

Epoch 1/1 : || 0.00% [0/1 00:00<00:00]

Epoch 1/1 : || 100.00% [1/1 00:00<00:00 2.3641]

Epoch 1/1 :                                    

Epoch 1/1 : || 0.00% [0/1 00:00<00:00]

Epoch 1/1 : || 100.00% [1/1 00:00<00:00]

Epoch 1/1 :                             
1         2.364101    398398.093750
Total time: 00:00
ok
test_torch_tensor (test_fastai.TestFastAI) ... ok
test_tokenize (test_fasttext.TestFastText) ... ok
test_fit (test_fbprophet.TestFbProphet) ... INFO:fbprophet.forecaster:Disabling yearly seasonality. Run prophet with yearly_seasonality=True to override this.
INFO:fbprophet.forecaster:Disabling weekly seasonality. Run prophet with weekly_seasonality=True to override this.
INFO:fbprophet.forecaster:Disabling daily seasonality. Run prophet with daily_seasonality=True to override this.
INFO:fbprophet.forecaster:n_changepoints greater than number of observations.Using 0.0.
/opt/conda/lib/python3.6/site-packages/pystan/misc.py:454: DeprecationWarning:

inspect.getargspec() is deprecated, use inspect.signature() or inspect.getfullargspec()


Gradient evaluation took 1.8e-05 seconds
1000 transitions using 10 leapfrog steps per transition would take 0.18 seconds.
Adjust your expectations accordingly!


WARNING: No variance estimation is
         performed for num_warmup < 20

Iteration: 1 / 1 [100%]  (Sampling)

 Elapsed Time: 1e-06 seconds (Warm-up)
               0.000141 seconds (Sampling)
               0.000142 seconds (Total)


Gradient evaluation took 1.6e-05 seconds
1000 transitions using 10 leapfrog steps per transition would take 0.16 seconds.
Adjust your expectations accordingly!


WARNING: No variance estimation is
         performed for num_warmup < 20

Iteration: 1 / 1 [100%]  (Sampling)

 Elapsed Time: 1e-06 seconds (Warm-up)
               5.1e-05 seconds (Sampling)
               5.2e-05 seconds (Total)


Gradient evaluation took 7e-06 seconds
1000 transitions using 10 leapfrog steps per transition would take 0.07 seconds.
Adjust your expectations accordingly!


WARNING: No variance estimation is
         performed for num_warmup < 20

Iteration: 1 / 1 [100%]  (Sampling)

 Elapsed Time: 2e-06 seconds (Warm-up)
               3.7e-05 seconds (Sampling)
               3.9e-05 seconds (Total)


Gradient evaluation took 1.5e-05 seconds
1000 transitions using 10 leapfrog steps per transition would take 0.15 seconds.
Adjust your expectations accordingly!


WARNING: No variance estimation is
         performed for num_warmup < 20

Iteration: 1 / 1 [100%]  (Sampling)

 Elapsed Time: 1e-06 seconds (Warm-up)
               3.1e-05 seconds (Sampling)
               3.2e-05 seconds (Total)

WARNING:pystan:n_eff / iter below 0.001 indicates that the effective sample size has likely been overestimated
WARNING:pystan:Rhat above 1.1 or below 0.9 indicates that the chains very likely have not mixed
WARNING:pystan:3 of 4 iterations ended with a divergence (75 %).
WARNING:pystan:Try running with adapt_delta larger than 0.8 to remove the divergences.
ok
test_read (test_fiona.TestFiona) ... ok
test_dictionary (test_gensim.TestGensim) ... INFO:gensim.corpora.dictionary:adding document #0 to Dictionary(0 unique tokens: [])
INFO:gensim.corpora.dictionary:built Dictionary(2 unique tokens: ['ipsum', 'lorem']) from 1 documents (total 2 corpus positions)
ok
test_read (test_geopandas.TestGeopandas) ... INFO:fiona.ogrext:Failed to auto identify EPSG: 7
ok
test_viz (test_geoviews.TestGeoviews) ... WARNING:root:main: Option 'height' for Feature type not valid for selected backend ('matplotlib'). Option only applies to following backends: ['bokeh']
ok
test_init_read (test_h2o.TestH2o) ... Checking whether there is an H2O instance running at http://localhost:54321..... not found.
Attempting to start a local H2O server...
  Java Version: openjdk version "1.8.0_181"; OpenJDK Runtime Environment (build 1.8.0_181-8u181-b13-2~deb9u1-b13); OpenJDK 64-Bit Server VM (build 25.181-b13, mixed mode)
  Starting server from /opt/conda/lib/python3.6/site-packages/h2o/backend/bin/h2o.jar
  Ice root: /tmp/tmphdkulls2
  JVM stdout: /tmp/tmphdkulls2/h2o_unknownUser_started_from_python.out
  JVM stderr: /tmp/tmphdkulls2/h2o_unknownUser_started_from_python.err
/opt/conda/lib/python3.6/site-packages/h2o/backend/server.py:138: ResourceWarning:

unclosed file <_io.TextIOWrapper name='/tmp/tmphdkulls2/h2o_unknownUser_started_from_python.out' mode='w' encoding='UTF-8'>

/opt/conda/lib/python3.6/site-packages/h2o/backend/server.py:138: ResourceWarning:

unclosed file <_io.TextIOWrapper name='/tmp/tmphdkulls2/h2o_unknownUser_started_from_python.err' mode='w' encoding='UTF-8'>

  Server is running at http://127.0.0.1:54321
Connecting to H2O server at http://127.0.0.1:54321... successful.
--------------------------  ---------------------------------------------------
H2O cluster uptime:         02 secs
H2O cluster timezone:       Etc/UTC
H2O data parsing timezone:  UTC
H2O cluster version:        3.22.1.4
H2O cluster version age:    11 days
H2O cluster name:           H2O_from_python_unknownUser_vqnw6r
H2O cluster total nodes:    1
H2O cluster free memory:    441 Mb
H2O cluster total cores:    2
H2O cluster allowed cores:  2
H2O cluster status:         accepting new members, healthy
H2O connection url:         http://127.0.0.1:54321
H2O connection proxy:
H2O internal security:      False
H2O API Extensions:         Amazon S3, XGBoost, Algos, AutoML, Core V3, Core V4
Python version:             3.6.6 final
--------------------------  ---------------------------------------------------

Parse progress: |        |   0%










H2O session _sid_98c8 closed.
ok
test_preprocessing (test_hep_ml.TestHepML) ... ok
test_curve (test_holoviews.TestHoloviews) ... ok
test_basic (test_imports.TestImport) ... ok
test_text_split (test_jieba.TestJieba) ... Building prefix dict from the default dictionary ...
DEBUG:jieba:Building prefix dict from the default dictionary ...
Loading model from cache /tmp/jieba.cache
DEBUG:jieba:Loading model from cache /tmp/jieba.cache
Loading model cost 0.972 seconds.
DEBUG:jieba:Loading model cost 0.972 seconds.
Prefix dict has been built succesfully.
DEBUG:jieba:Prefix dict has been built succesfully.
ok
test_jupytertensorboard (test_jupytertensorboard.TestJupyterTensorBoard) ... [NbConvertApp] Converting notebook /input/tests/data/jupyter_tensorboard.ipynb to html
[NbConvertApp] Executing notebook with kernel: python3
ok
test_conv2d (test_keras.TestKeras) ... /opt/conda/lib/python3.6/site-packages/numpy/lib/type_check.py:546: DeprecationWarning:

np.asscalar(a) is deprecated since NumPy v1.16, use a.item() instead

Epoch 1/1
2019-02-27 02:34:02.237804: W tensorflow/core/framework/allocator.cc:122] Allocation of 31719424 exceeds 10% of system memory.
2019-02-27 02:34:02.261496: W tensorflow/core/framework/allocator.cc:122] Allocation of 31719424 exceeds 10% of system memory.
2019-02-27 02:34:02.283480: W tensorflow/core/framework/allocator.cc:122] Allocation of 31719424 exceeds 10% of system memory.
2019-02-27 02:34:02.334113: W tensorflow/core/framework/allocator.cc:122] Allocation of 31719424 exceeds 10% of system memory.
2019-02-27 02:34:02.382865: W tensorflow/core/framework/allocator.cc:122] Allocation of 31719424 exceeds 10% of system memory.

 32/100 [========>.....................] - ETA: 5s - loss: 2.3118�����������������������������������������������������������������
 64/100 [==================>...........] - ETA: 2s - loss: 2.3421�����������������������������������������������������������������
 96/100 [===========================>..] - ETA: 0s - loss: 2.3679�����������������������������������������������������������������
100/100 [==============================] - 6s 57ms/step - loss: 2.3618

20/20 [==============================] - 0s 16ms/step
ok
test_train (test_keras.TestKeras) ... Epoch 1/1

 32/100 [========>.....................] - ETA: 0s - loss: 13.5997 - acc: 0.1562��������������������������������������������������������������������������������
100/100 [==============================] - 0s 1ms/step - loss: 14.5936 - acc: 0.0900
ok
test_init (test_kmapper.TestKMapper) ... ok
test_network_definition (test_lasagne.TestLasagne) ... ok
test_check (test_learntools.TestLearnTools) ... <IPython.core.display.Javascript object>
Check: When you've updated the starter code, `check()` will tell you whether your code is correct. Remember, you must create the following variable: `color`
ok
test_simple (test_lightgbm.TestLightgbm) ... [LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[1]	valid_0's l2: 2.17563	valid_0's auc: 1
Training until validation scores don't improve for 1 rounds.
Did not meet early stopping. Best iteration is:
[1]	valid_0's l2: 2.17563	valid_0's auc: 1
ok
test_basemap (test_matplotlib.TestMatplotlib) ... /opt/conda/lib/python3.6/site-packages/mpl_toolkits/basemap/__init__.py:1299: DeprecationWarning:

The binary mode of fromstring is deprecated, as it behaves surprisingly on unicode inputs. Use frombuffer instead

ok
test_plot (test_matplotlib.TestMatplotlib) ... ok
test_tokenize (test_nltk.TestNLTK) ... ok
test_array (test_numpy.TestNumpy) ... ok
test_mkl (test_numpy.TestNumpy) ... ok
test_pycuda (test_nvidia.TestNvidia) ... skipped 'Not running GPU tests'
test_system_management_interface (test_nvidia.TestNvidia) ... skipped 'Not running GPU tests'
test_imread (test_opencv.TestOpenCV) ... ok
test_read_csv (test_pandas.TestPandas) ... ok
test_read_feather (test_pandas.TestPandas) ... /opt/conda/lib/python3.6/importlib/_bootstrap.py:219: ImportWarning:

can't resolve package from __spec__ or __package__, falling back on __name__ and __path__

/opt/conda/lib/python3.6/site-packages/pandas/io/feather_format.py:112: FutureWarning:

`nthreads` argument is deprecated, pass `use_threads` instead

ok
test_figure (test_plotly.TestPlotly) ... ok
test_vector (test_pyfasttext.TestPyFasttext) ... 
Read 0M words
Number of words:  29
Number of labels: 0

Progress: 0.0%  words/sec/thread: 73699  lr: 0.700000  loss: 0.000000  eta: -596523h-14m 
Progress: 100.0%  words/sec/thread: 65301  lr: 0.000000  loss: 0.000000  eta: 0h0m 
ok
test_kalman_filter (test_pykalman.TestPyKalman) ... /opt/conda/lib/python3.6/site-packages/pykalman/utils.py:111: DeprecationWarning:

inspect.getargspec() is deprecated, use inspect.signature() or inspect.getfullargspec()

ok
test_kalman_missing (test_pykalman.TestPyKalman) ... ok
test_online_update (test_pykalman.TestPyKalman) ... ok
test_robust_sqrt (test_pykalman.TestPyKalman) ... ok
test_unscented_kalman (test_pykalman.TestPyKalman) ... ok
test_has_vectors (test_pymagnitude.TestPyMagnitude) ... ok
test_vector_query (test_pymagnitude.TestPyMagnitude) ... ok
test_vector_search (test_pymagnitude.TestPyMagnitude) ... ok
test_tokenize (test_pytext.TestPyText) ... ok
test_cuda_nn (test_pytorch.TestPyTorch) ... skipped 'Not running GPU tests'
test_gpu_computation (test_pytorch.TestPyTorch) ... skipped 'Not running GPU tests'
test_nn (test_pytorch.TestPyTorch) ... ok
test_classifier (test_rgf.TestRGF) ... ok
test_option (test_seaborn.TestSeaborn) ... ok
test_version (test_seaborn.TestSeaborn) ... ok
test_init (test_shap.TestShap) ... <IPython.core.display.HTML object>
ok
test_geometry (test_shapely.TestShapely) ... ok
test_filter (test_skimage.TestSkImage) ... ok
test_linearn_classifier (test_sklearn.TestSklearn) ... ok
test_random_forest_classifier (test_sklearn.TestSklearn) ... /opt/conda/lib/python3.6/site-packages/sklearn/ensemble/forest.py:246: FutureWarning:

The default value of n_estimators will change from 10 in version 0.20 to 100 in 0.22.

ok
test_addition (test_tensorflow.TestTensorflow) ... ok
test_conv2d (test_tensorflow.TestTensorflow) ... ok
test_cudnn_lstm (test_tensorflow.TestTensorflow) ... skipped 'Not running GPU tests'
test_gpu (test_tensorflow.TestTensorflow) ... skipped 'Not running GPU tests'
test_addition (test_theano.TestTheano) ... ok
test_extract_feature (test_tsfresh.TestTsFresh) ... /opt/conda/lib/python3.6/site-packages/tsfresh/feature_extraction/settings.py:106: DeprecationWarning:

inspect.getargspec() is deprecated, use inspect.signature() or inspect.getfullargspec()


Feature Extraction:   0% 0/2 [00:00<?, ?it/s]
Feature Extraction:  50% 1/2 [00:00<00:00,  4.94it/s]
Feature Extraction: 100% 2/2 [00:00<00:00,  6.94it/s]
ok
test_get_access_token_succeeds (test_user_secrets.TestUserSecrets) ... 127.0.0.1 - - [27/Feb/2019 02:34:18] "POST /requests/GetUserSecretRequest HTTP/1.1" 200 -
ok
test_no_token_fails (test_user_secrets.TestUserSecrets) ... ok
test_basic (test_vowpalwabbit.TestVowpalwabbit) ... ok
test_import (test_vtk.TestVTK) ... ok
test_wordbatch (test_wordbatch.TestWordBatch) ... ok
test_classifier (test_xgboost.TestXGBoost) ... ok
test_xvfb (test_xvfbwrapper.TestXvfbwrapper) ... ok

----------------------------------------------------------------------
Ran 77 tests in 39.930s

OK (skipped=6)

@sebbov
Copy link
Contributor

sebbov commented Feb 27, 2019

Did you answer the following general requests? Sorry if I missed them or they are still coming.

  • Have you tested it end-to-end in Kaggle Kernels? For both CPU and GPU? Can you include the steps and results in the PR description please.
  • Please make sure our internal worker integration tests passes with the update to this image.

@emzeq
Copy link
Contributor Author

emzeq commented Feb 28, 2019 •
edited
Loading

GPU Test output:

~/docker-python$ ./test -g
+ rm -rf /tmp/python-build
+ docker rm jupyter_test
Error: No such container: jupyter_test
+ true
+ mkdir -p /tmp/python-build/tmp
+ mkdir -p /tmp/python-build/devshm
+ mkdir -p /tmp/python-build/working
+ docker run -d --name=jupyter_test --read-only --net=none -e HOME=/tmp -v /home/aagundez/docker-python:/input:ro -v /tmp/python-build/working:/working -w=/working -v /tmp/python-build/tmp:/tmp -v /tmp/python-build/devshm:/dev/shm kaggle/python-gpu-build jupyter notebook --allow-root '--ip=*'
884771d787fdb58643de894ccca1c18f769c321ece1b4eb7578d1f2b7c634aa8
+ sleep 3
+ docker kill jupyter_test
jupyter_test
+ docker rm jupyter_test
jupyter_test
+ docker run --rm -t --read-only --net=none -e HOME=/tmp -e KAGGLE_DATA_PROXY_TOKEN=test-key -e KAGGLE_USER_SECRETS_TOKEN_KEY=test-secrets-key -e KAGGLE_URL_BASE=http://127.0.0.1:8001 -e KAGGLE_DATA_PROXY_URL=http://127.0.0.1:8000 -e KAGGLE_DATA_PROXY_PROJECT=test -v /home/aagundez/docker-python:/input:ro -v /tmp/python-build/working:/working -v /tmp/python-build/tmp:/tmp -v /tmp/python-build/devshm:/dev/shm -w=/working -v /tmp/empty_dir:/usr/local/cuda/lib64/stubs:ro kaggle/python-gpu-build /bin/bash -c 'python -m unittest discover -s /input/tests -p test*.py -v'
/opt/conda/lib/python3.6/site-packages/distributed/utils.py:134: RuntimeWarning: Couldn't detect a suitable IP address for reaching '8.8.8.8', defaulting to '127.0.0.1': [Errno 101] Network is unreachable
  % (host, default, e), RuntimeWarning)
[   INFO   ] MusicExtractorSVM: no classifier models were configured by default
/opt/conda/lib/python3.6/site-packages/scipy/sparse/sparsetools.py:21: DeprecationWarning: `scipy.sparse.sparsetools` is deprecated!
scipy.sparse.sparsetools is a private module for scipy.sparse, and should not be used.
  _deprecated()
INFO:gensim.summarization.textcleaner:'pattern' package not found; tag filters are not available for English
/opt/conda/lib/python3.6/site-packages/tabulate-0.8.3-py3.6.egg/tabulate.py:192: DeprecationWarning: invalid escape sequence \h
/opt/conda/lib/python3.6/site-packages/tabulate-0.8.3-py3.6.egg/tabulate.py:415: DeprecationWarning: invalid escape sequence \[
/opt/conda/lib/python3.6/site-packages/tabulate-0.8.3-py3.6.egg/tabulate.py:1258: DeprecationWarning: invalid escape sequence \e
/opt/conda/lib/python3.6/site-packages/tabulate-0.8.3-py3.6.egg/tabulate.py:192: DeprecationWarning: invalid escape sequence \h
/opt/conda/lib/python3.6/site-packages/tabulate-0.8.3-py3.6.egg/tabulate.py:415: DeprecationWarning: invalid escape sequence \[
/opt/conda/lib/python3.6/site-packages/tabulate-0.8.3-py3.6.egg/tabulate.py:1258: DeprecationWarning: invalid escape sequence \e
Using TensorFlow backend.
/opt/conda/lib/python3.6/site-packages/tensorflow/python/util/tf_inspect.py:75: DeprecationWarning: inspect.getargspec() is deprecated, use inspect.signature() or inspect.getfullargspec()
  return _inspect.getargspec(target)
WARNING:theano.tensor.blas:Using NumPy C-API based implementation for BLAS functions.
/opt/conda/lib/python3.6/site-packages/theano/tests/main.py:6: DeprecationWarning: Importing from numpy.testing.nosetester is deprecated since 1.15.0, import from numpy.testing instead.
  from numpy.testing.nosetester import NoseTester
/opt/conda/lib/python3.6/site-packages/shap/plots/colors.py:35: DeprecationWarning:

object of type <class 'float'> cannot be safely interpreted as an integer.

/opt/conda/lib/python3.6/site-packages/tensorflow/python/util/tf_inspect.py:75: DeprecationWarning:

inspect.getargspec() is deprecated, use inspect.signature() or inspect.getfullargspec()

test_no_project_with_connected_account (test_bigquery.TestBigQuery) ... INFO:google.auth.compute_engine._metadata:Compute Engine Metadata server unavailable onattempt 1 of 3
INFO:google.auth.compute_engine._metadata:Compute Engine Metadata server unavailable onattempt 2 of 3
INFO:google.auth.compute_engine._metadata:Compute Engine Metadata server unavailable onattempt 3 of 3
ok
test_project_with_connected_account (test_bigquery.TestBigQuery) ... Calling Kaggle.UserSecrets to refresh token.
ok
test_proxy_no_project (test_bigquery.TestBigQuery) ... Using Kaggle's public dataset BigQuery integration.
127.0.0.1 - - [28/Feb/2019 06:40:23] "GET /bigquery/v2/projects/test/datasets HTTP/1.1" 200 -
ok
test_proxy_using_library (test_bigquery.TestBigQuery) ... 127.0.0.1 - - [28/Feb/2019 06:40:24] "GET /bigquery/v2/projects/test/datasets HTTP/1.1" 200 -
ok
test_simultaneous_clients (test_bigquery.TestBigQuery) ... Using Kaggle's public dataset BigQuery integration.
127.0.0.1 - - [28/Feb/2019 06:40:25] "GET /bigquery/v2/projects/test/datasets HTTP/1.1" 200 -
Calling Kaggle.UserSecrets to refresh token.
127.0.0.1 - - [28/Feb/2019 06:40:26] "GET /bigquery/v2/projects/test/datasets HTTP/1.1" 200 -
ok
test_figure (test_bokeh.TestBokeh) ... ok
test_projection (test_cartopy.TestCartopy) ... ok
test_pipeline (test_datashader.TestDatashader) ... ok
test_renderer (test_dipy.TestDipy) ... ok
test_windowing (test_essentia.TestEssentia) ... ok
test_has_version (test_fastai.TestFastAI) ... ok
test_partition (test_fastai.TestFastAI) ... ok
test_tabular (test_fastai.TestFastAI) ... epoch     train_loss  valid_loss  time
1         2.447515    1.778500    00:00
Total time: 00:00
ok
test_torch_tensor (test_fastai.TestFastAI) ... ok
test_tokenize (test_fasttext.TestFastText) ... ok
test_fit (test_fbprophet.TestFbProphet) ... INFO:fbprophet.forecaster:Disabling yearly seasonality. Run prophet with yearly_seasonality=True to override this.
INFO:fbprophet.forecaster:Disabling weekly seasonality. Run prophet with weekly_seasonality=True to override this.
INFO:fbprophet.forecaster:Disabling daily seasonality. Run prophet with daily_seasonality=True to override this.
INFO:fbprophet.forecaster:n_changepoints greater than number of observations.Using 0.0.
/opt/conda/lib/python3.6/site-packages/pystan/misc.py:454: DeprecationWarning:

inspect.getargspec() is deprecated, use inspect.signature() or inspect.getfullargspec()


Gradient evaluation took 2.8e-05 seconds
1000 transitions using 10 leapfrog steps per transition would take 0.28 seconds.
Adjust your expectations accordingly!


WARNING: No variance estimation is
         performed for num_warmup < 20

Iteration: 1 / 1 [100%]  (Sampling)

 Elapsed Time: 2e-06 seconds (Warm-up)
               5.6e-05 seconds (Sampling)
               5.8e-05 seconds (Total)


Gradient evaluation took 1.3e-05 seconds
1000 transitions using 10 leapfrog steps per transition would take 0.13 seconds.
Adjust your expectations accordingly!


WARNING: No variance estimation is
         performed for num_warmup < 20

Iteration: 1 / 1 [100%]  (Sampling)

 Elapsed Time: 2e-06 seconds (Warm-up)
               5.6e-05 seconds (Sampling)
               5.8e-05 seconds (Total)


Gradient evaluation took 3.9e-05 seconds
1000 transitions using 10 leapfrog steps per transition would take 0.39 seconds.
Adjust your expectations accordingly!


WARNING: No variance estimation is

Gradient evaluation took 1.2e-05 seconds
1000 transitions using 10 leapfrog steps per transition would take 0.12 seconds.
Adjust your expectations accordingly!


WARNING: No variance estimation is
         performed for num_warmup < 20

Iteration: 1 / 1 [100%]  (Sampling)

 Elapsed Time: 2e-06 seconds (Warm-up)
               6.7e-05 seconds (Sampling)
               6.9e-05 seconds (Total)

         performed for num_warmup < 20

Iteration: 1 / 1 [100%]  (Sampling)

 Elapsed Time: 2e-06 seconds (Warm-up)
               9.3e-05 seconds (Sampling)
               9.5e-05 seconds (Total)

WARNING:pystan:n_eff / iter below 0.001 indicates that the effective sample size has likely been overestimated
WARNING:pystan:Rhat above 1.1 or below 0.9 indicates that the chains very likely have not mixed
WARNING:pystan:4 of 4 iterations ended with a divergence (100 %).
WARNING:pystan:Try running with adapt_delta larger than 0.8 to remove the divergences.
ok
test_read (test_fiona.TestFiona) ... ok
test_dictionary (test_gensim.TestGensim) ... INFO:gensim.corpora.dictionary:adding document #0 to Dictionary(0 unique tokens: [])
INFO:gensim.corpora.dictionary:built Dictionary(2 unique tokens: ['ipsum', 'lorem']) from 1 documents (total 2 corpus positions)
ok
test_read (test_geopandas.TestGeopandas) ... INFO:fiona.ogrext:Failed to auto identify EPSG: 7
ok
test_viz (test_geoviews.TestGeoviews) ... WARNING:root:main: Option 'height' for Feature type not valid for selected backend ('matplotlib'). Option only applies to following backends: ['bokeh']
ok
test_init_read (test_h2o.TestH2o) ... Checking whether there is an H2O instance running at http://localhost:54321..... not found.
Attempting to start a local H2O server...
  Java Version: openjdk version "1.8.0_181"; OpenJDK Runtime Environment (build 1.8.0_181-8u181-b13-2~deb9u1-b13); OpenJDK 64-Bit Server VM (build 25.181-b13, mixed mode)
  Starting server from /opt/conda/lib/python3.6/site-packages/h2o/backend/bin/h2o.jar
  Ice root: /tmp/tmpizqsw2a5
  JVM stdout: /tmp/tmpizqsw2a5/h2o_unknownUser_started_from_python.out
  JVM stderr: /tmp/tmpizqsw2a5/h2o_unknownUser_started_from_python.err
/opt/conda/lib/python3.6/site-packages/h2o/backend/server.py:138: ResourceWarning:

unclosed file <_io.TextIOWrapper name='/tmp/tmpizqsw2a5/h2o_unknownUser_started_from_python.out' mode='w' encoding='UTF-8'>

/opt/conda/lib/python3.6/site-packages/h2o/backend/server.py:138: ResourceWarning:

unclosed file <_io.TextIOWrapper name='/tmp/tmpizqsw2a5/h2o_unknownUser_started_from_python.err' mode='w' encoding='UTF-8'>

  Server is running at http://127.0.0.1:54321
Connecting to H2O server at http://127.0.0.1:54321... successful.
--------------------------  ---------------------------------------------------
H2O cluster uptime:         02 secs
H2O cluster timezone:       Etc/UTC
H2O data parsing timezone:  UTC
H2O cluster version:        3.22.1.4
H2O cluster version age:    12 days
H2O cluster name:           H2O_from_python_unknownUser_6leaet
H2O cluster total nodes:    1
H2O cluster free memory:    3.484 Gb
H2O cluster total cores:    2
H2O cluster allowed cores:  2
H2O cluster status:         accepting new members, healthy
H2O connection url:         http://127.0.0.1:54321
H2O connection proxy:
H2O internal security:      False
H2O API Extensions:         Amazon S3, XGBoost, Algos, AutoML, Core V3, Core V4
Python version:             3.6.6 final
--------------------------  ---------------------------------------------------
Parse progress: |█████████████████████████████████████████████████████████████████████████████| 100%
H2O session _sid_aa85 closed.
ok
test_preprocessing (test_hep_ml.TestHepML) ... ok
test_curve (test_holoviews.TestHoloviews) ... ok
test_basic (test_imports.TestImport) ... ok
test_text_split (test_jieba.TestJieba) ... Building prefix dict from the default dictionary ...
DEBUG:jieba:Building prefix dict from the default dictionary ...
Dumping model to file cache /tmp/jieba.cache
DEBUG:jieba:Dumping model to file cache /tmp/jieba.cache
Loading model cost 1.021 seconds.
DEBUG:jieba:Loading model cost 1.021 seconds.
Prefix dict has been built succesfully.
DEBUG:jieba:Prefix dict has been built succesfully.
ok
test_jupytertensorboard (test_jupytertensorboard.TestJupyterTensorBoard) ... [NbConvertApp] Converting notebook /input/tests/data/jupyter_tensorboard.ipynb to html
[NbConvertApp] Executing notebook with kernel: python3
ok
test_conv2d (test_keras.TestKeras) ... /opt/conda/lib/python3.6/site-packages/numpy/lib/type_check.py:546: DeprecationWarning:

np.asscalar(a) is deprecated since NumPy v1.16, use a.item() instead

Epoch 1/1
2019-02-28 06:40:50.298467: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:964] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2019-02-28 06:40:50.299090: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1432] Found device 0 with properties:
name: Tesla K80 major: 3 minor: 7 memoryClockRate(GHz): 0.8235
pciBusID: 0000:00:04.0
totalMemory: 11.17GiB freeMemory: 10.84GiB
2019-02-28 06:40:50.299135: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1511] Adding visible gpu devices: 0
2019-02-28 06:40:59.144708: I tensorflow/core/common_runtime/gpu/gpu_device.cc:982] Device interconnect StreamExecutor with strength 1 edge matrix:
2019-02-28 06:40:59.144762: I tensorflow/core/common_runtime/gpu/gpu_device.cc:988]      0
2019-02-28 06:40:59.144925: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1001] 0:   N
2019-02-28 06:40:59.145591: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1115] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 10506 MB memory) -> physical GPU (device: 0, name: Tesla K80, pci bus id: 0000:00:04.0, compute capability: 3.7)
100/100 [==============================] - 10s 103ms/step - loss: 2.3472
20/20 [==============================] - 0s 9ms/step
ok
test_train (test_keras.TestKeras) ... Epoch 1/1
100/100 [==============================] - 0s 2ms/step - loss: 13.9000 - acc: 0.1300
ok
test_init (test_kmapper.TestKMapper) ... ok
test_network_definition (test_lasagne.TestLasagne) ... ok
test_check (test_learntools.TestLearnTools) ... <IPython.core.display.Javascript object>
Check: When you've updated the starter code, `check()` will tell you whether your code is correct. Remember, you must create the following variable: `color`
ok
test_simple (test_lightgbm.TestLightgbm) ... [LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[1]     valid_0's auc: 1        valid_0's l2: 2.17562
Training until validation scores don't improve for 1 rounds.
Did not meet early stopping. Best iteration is:
[1]     valid_0's auc: 1        valid_0's l2: 2.17562
ok
test_basemap (test_matplotlib.TestMatplotlib) ... /opt/conda/lib/python3.6/site-packages/mpl_toolkits/basemap/__init__.py:1299: DeprecationWarning:

The binary mode of fromstring is deprecated, as it behaves surprisingly on unicode inputs. Use frombuffer instead

ok
test_plot (test_matplotlib.TestMatplotlib) ... ok
test_tokenize (test_nltk.TestNLTK) ... ok
test_array (test_numpy.TestNumpy) ... ok
test_mkl (test_numpy.TestNumpy) ... ok
test_pycuda (test_nvidia.TestNvidia) ... ok
test_system_management_interface (test_nvidia.TestNvidia) ... ok
test_imread (test_opencv.TestOpenCV) ... ok
test_read_csv (test_pandas.TestPandas) ... ok
test_read_feather (test_pandas.TestPandas) ... /opt/conda/lib/python3.6/importlib/_bootstrap.py:219: ImportWarning:

can't resolve package from __spec__ or __package__, falling back on __name__ and __path__

/opt/conda/lib/python3.6/site-packages/pandas/io/feather_format.py:112: FutureWarning:

`nthreads` argument is deprecated, pass `use_threads` instead

ok
test_figure (test_plotly.TestPlotly) ... ok
Read 0M wordstest_pyfasttext.TestPyFasttext) ...
Number of words:  29
Number of labels: 0
Progress: 100.0%  words/sec/thread: 471888  lr: 0.000000  loss: 0.000000  eta: 0h0m h-14m
ok
test_kalman_filter (test_pykalman.TestPyKalman) ... /opt/conda/lib/python3.6/site-packages/pykalman/utils.py:111: DeprecationWarning:

inspect.getargspec() is deprecated, use inspect.signature() or inspect.getfullargspec()

ok
test_kalman_missing (test_pykalman.TestPyKalman) ... ok
test_online_update (test_pykalman.TestPyKalman) ... ok
test_robust_sqrt (test_pykalman.TestPyKalman) ... ok
test_unscented_kalman (test_pykalman.TestPyKalman) ... ok
test_has_vectors (test_pymagnitude.TestPyMagnitude) ... ok
test_vector_query (test_pymagnitude.TestPyMagnitude) ... ok
test_vector_search (test_pymagnitude.TestPyMagnitude) ... ok
test_tokenize (test_pytext.TestPyText) ... ok
test_cuda_nn (test_pytorch.TestPyTorch) ... ok
test_gpu_computation (test_pytorch.TestPyTorch) ... ok
test_nn (test_pytorch.TestPyTorch) ... ok
test_classifier (test_rgf.TestRGF) ... ok
test_option (test_seaborn.TestSeaborn) ... ok
test_version (test_seaborn.TestSeaborn) ... ok
test_init (test_shap.TestShap) ... <IPython.core.display.HTML object>
ok
test_geometry (test_shapely.TestShapely) ... ok
test_filter (test_skimage.TestSkImage) ... ok
test_linearn_classifier (test_sklearn.TestSklearn) ... ok
test_random_forest_classifier (test_sklearn.TestSklearn) ... /opt/conda/lib/python3.6/site-packages/sklearn/ensemble/forest.py:246: FutureWarning:

The default value of n_estimators will change from 10 in version 0.20 to 100 in 0.22.

ok
test_addition (test_tensorflow.TestTensorflow) ... 2019-02-28 06:41:08.003616: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1511] Adding visible gpu devices: 0
2019-02-28 06:41:08.011333: I tensorflow/core/common_runtime/gpu/gpu_device.cc:982] Device interconnect StreamExecutor with strength 1 edge matrix:
2019-02-28 06:41:08.011430: I tensorflow/core/common_runtime/gpu/gpu_device.cc:988]      0
2019-02-28 06:41:08.011466: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1001] 0:   N
2019-02-28 06:41:08.011786: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1115] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 10506 MB memory) -> physical GPU (device: 0, name: Tesla K80, pci bus id: 0000:00:04.0, compute capability: 3.7)
ok
test_conv2d (test_tensorflow.TestTensorflow) ... 2019-02-28 06:41:08.180754: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1511] Adding visible gpu devices: 0
2019-02-28 06:41:08.187303: I tensorflow/core/common_runtime/gpu/gpu_device.cc:982] Device interconnect StreamExecutor with strength 1 edge matrix:
2019-02-28 06:41:08.187415: I tensorflow/core/common_runtime/gpu/gpu_device.cc:988]      0
2019-02-28 06:41:08.187499: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1001] 0:   N
2019-02-28 06:41:08.187796: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1115] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 10506 MB memory) -> physical GPU (device: 0, name: Tesla K80, pci bus id: 0000:00:04.0, compute capability: 3.7)
ok
test_cudnn_lstm (test_tensorflow.TestTensorflow) ... 2019-02-28 06:41:08.890980: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1511] Adding visible gpu devices: 0
2019-02-28 06:41:08.891243: I tensorflow/core/common_runtime/gpu/gpu_device.cc:982] Device interconnect StreamExecutor with strength 1 edge matrix:
2019-02-28 06:41:08.891367: I tensorflow/core/common_runtime/gpu/gpu_device.cc:988]      0
2019-02-28 06:41:08.891518: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1001] 0:   N
2019-02-28 06:41:08.891845: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1115] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 10506 MB memory) -> physical GPU (device: 0, name: Tesla K80, pci bus id: 0000:00:04.0, compute capability: 3.7)
ok
test_gpu (test_tensorflow.TestTensorflow) ... 2019-02-28 06:41:09.800543: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1511] Adding visible gpu devices: 0
2019-02-28 06:41:09.800627: I tensorflow/core/common_runtime/gpu/gpu_device.cc:982] Device interconnect StreamExecutor with strength 1 edge matrix:
2019-02-28 06:41:09.800658: I tensorflow/core/common_runtime/gpu/gpu_device.cc:988]      0
2019-02-28 06:41:09.800704: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1001] 0:   N
2019-02-28 06:41:09.800936: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1115] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 10506 MB memory) -> physical GPU (device: 0, name: Tesla K80, pci bus id: 0000:00:04.0, compute capability: 3.7)
ok
test_addition (test_theano.TestTheano) ... ok
test_extract_feature (test_tsfresh.TestTsFresh) ... /opt/conda/lib/python3.6/site-packages/tsfresh/feature_extraction/settings.py:106: DeprecationWarning:

inspect.getargspec() is deprecated, use inspect.signature() or inspect.getfullargspec()

Feature Extraction: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00,  7.70it/s]
ok
test_get_access_token_succeeds (test_user_secrets.TestUserSecrets) ... 127.0.0.1 - - [28/Feb/2019 06:41:12] "POST /requests/GetUserSecretRequest HTTP/1.1" 200 -
ok
test_no_token_fails (test_user_secrets.TestUserSecrets) ... ok
test_basic (test_vowpalwabbit.TestVowpalwabbit) ... ok
test_import (test_vtk.TestVTK) ... ok
test_wordbatch (test_wordbatch.TestWordBatch) ... ok
test_classifier (test_xgboost.TestXGBoost) ... ok
test_xvfb (test_xvfbwrapper.TestXvfbwrapper) ... ok

----------------------------------------------------------------------
Ran 77 tests in 49.439s

OK

@emzeq
Copy link
Contributor Author

emzeq commented Feb 28, 2019

@rosbo @sebbov Anything else needed to merge this?

@sebbov
Copy link
Contributor

sebbov commented Feb 28, 2019

LGTM, thanks!

@rosbo
Copy link
Contributor

rosbo commented Feb 28, 2019

Merge at will :)

@emzeq emzeq merged commit a6ba32e into master Feb 28, 2019
@emzeq emzeq deleted the tensorboard branch February 28, 2019 22:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@rosbo rosbo rosbo approved these changes

@corrieann corrieann Awaiting requested review from corrieann

+1 more reviewer

@sebbov sebbov sebbov left review comments

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@emzeq @rosbo @sebbov