Dev (#20)

* sync json with neptune (#17) * added telegram bot for neptune-lib (#18)
neptune-ai · Feb 9, 2019 · c319269 · c319269
1 parent 3ed5613
commit c319269
Show file tree

Hide file tree

Showing 12 changed files with 620 additions and 3 deletions.
diff --git a/docs/conf.py b/docs/conf.py
@@ -18,7 +18,7 @@
 sys.path.insert(0, os.path.abspath('..'))
 
 # -- Imports mock -----------------------------------------------------]
-autodoc_mock_imports = ['altair', 'fastai', 'fastai.callbacks']
+autodoc_mock_imports = ['altair', 'fastai', 'fastai.callbacks', 'telegram']
 
 # -- Project information -----------------------------------------------------
 
@@ -27,9 +27,9 @@
 author = 'Neptune Dev Team'
 
 # The short X.Y version
-version = '0.1'
+version = '0.2'
 # The full version, including alpha/beta/rc tags
-release = '0.1.0'
+release = '0.2.0'
 
 # -- General configuration ---------------------------------------------------
 

diff --git a/docs/examples/examples_index.rst b/docs/examples/examples_index.rst
@@ -7,3 +7,5 @@
    Log matplotlib charts to neptune <log_matplotlib>
    Work with local notebooks <local_notebooks>
    Visualize project progress <project_progress>
+   Sync experiments with Neptune via json file <sync_with_json>
+   Telegram bot <telegram_bot>
diff --git a/docs/examples/sync_with_json.ipynb b/docs/examples/sync_with_json.ipynb
@@ -0,0 +1,70 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# Sync any experiment with Neptune\n",
+    "## Create .json file containing your experiment data\n",
+    "You can run experiment in any language or framework and simply create a \n",
+    "json with all the data that you care to save to neptune.\n",
+    "\n",
+    "For example your `experiment_data.json` could look like this:\n",
+    "\n",
+    "```json\n",
+    "    {\"name\": \"baseline\", \n",
+    "     \"parameters\": {\"lr\": 0.1, \n",
+    "                    \"batch_size\": 32\n",
+    "                   }, \n",
+    "     \"tags\": [\"base\", \"solution-1\", \"pytorch\"], \n",
+    "     \"channels\": {\"log_loss\": {\"x\": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10], \n",
+    "                               \"y\": [0, 1, 4, 9, 16, 25, 36, 49, 64, 81, 100]\n",
+    "                              }\n",
+    "                 }, \n",
+    "     \"properties\": {\"data_version\": \"version_1\"\n",
+    "                   }\n",
+    "    }\n",
+    "```\n",
+    "\n",
+    "## Sync with Neptune\n",
+    "Now all you need is to send this data to your Neptune project.\n",
+    "\n",
+    "```bash\n",
+    "python -m neptunecontrib.sync.with_json \\\n",
+    "--project_name neptune-ml/neptune-examples \\\n",
+    "--filepath experiment_data.json\n",
+    "\n",
+    "```\n",
+    "\n",
+    "## Explore your experiment in Neptune\n",
+    "Now your experiment data is safely logged to Neptune and you can easily explore it.\n",
+    "Go to the [experiment link](https://app.neptune.ml/o/neptune-ml/org/neptune-examples/e/NEP-506/charts) to see for yourself!\n",
+    "\n",
+    "![image_detail](https://gist.githubusercontent.com/jakubczakon/f754769a39ea6b8fa9728ede49b9165c/raw/18b476c5ba53b7f7c67f8c61eba1a22a98862f83/contrib_sync1.png)\n",
+    "\n",
+    "![image_chart](https://gist.githubusercontent.com/jakubczakon/f754769a39ea6b8fa9728ede49b9165c/raw/18b476c5ba53b7f7c67f8c61eba1a22a98862f83/contrib_sync2.png)"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "gap",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.6.8"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 2
+}
diff --git a/docs/examples/telegram_bot.ipynb b/docs/examples/telegram_bot.ipynb
@@ -0,0 +1,69 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# Telegram Bot\n",
+    "Inspired by the beautiful [blog post and package Telegrad](https://towardsdatascience.com/how-to-monitor-and-control-deep-learning-experiments-through-your-phone-35ef1704928d\n",
+    ") we have decided to write the telegram bot for Neptune!\n",
+    "\n",
+    "## Prerequisites\n",
+    "\n",
+    "### Create telegram bot api token\n",
+    "\n",
+    "* Open the Telegram app\n",
+    "* Search for the BotFather user\n",
+    "* Start a conversation with BotFather and click on start\n",
+    "* Send /newbot and follow the instructions on the screen\n",
+    "* Copy the bot token\n",
+    "\n",
+    "### Get neptune api token\n",
+    " * Run the following command:\n",
+    "\n",
+    "```bash\n",
+    "neptune account api-token get\n",
+    "```\n",
+    "\n",
+    " * Copy neptune token\n",
+    "\n",
+    "## Fire up your bot\n",
+    "Now all you need is to send this data to your Neptune project.\n",
+    "\n",
+    "```bash\n",
+    "python -m neptunecontrib.bots.telegram_bot \\\n",
+    "--telegram_api_token \"asdasfasdg23523rfasdfasd\" \\\n",
+    "--neptune_api_token \"asfasdgfaewae23423qwefdASFsa=\"\n",
+    "\n",
+    "```\n",
+    "\n",
+    "## Talk to your bot\n",
+    "\n",
+    "![image_bot1](https://gist.githubusercontent.com/jakubczakon/f754769a39ea6b8fa9728ede49b9165c/raw/2e6f056574d035c6d744fc5d89f1ed55bf155abf/contrib_bot1.png)\n",
+    "\n",
+    "![image_bot2](https://gist.githubusercontent.com/jakubczakon/f754769a39ea6b8fa9728ede49b9165c/raw/2e6f056574d035c6d744fc5d89f1ed55bf155abf/contrib_bot2.png)"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "neptunecontrib_py36",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.6.8"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 2
+}
diff --git a/docs/index.rst b/docs/index.rst
@@ -30,11 +30,13 @@ And the best thing is you can extend it yourself or... tell us to do it for you
 
    api.utils <user_guide/api/utils>
    hpo.utils <user_guide/hpo/utils>
+   bots.telegram_bot <user_guide/bots/telegram_bot>
    monitoring.notebooks <user_guide/monitoring/notebooks>
    monitoring.lightgbm <user_guide/monitoring/lightgbm>
    monitoring.fastai <user_guide/monitoring/fastai>
    monitoring.skopt <user_guide/monitoring/skopt>
    monitoring.utils <user_guide/monitoring/utils>
+   sync.with_json <user_guide/sync/with_json>
    viz.experiments <user_guide/viz/experiments>
    viz.projects <user_guide/viz/projects>
    viz.utils <user_guide/viz/utils>

diff --git a/docs/user_guide/bots/telegram_bot.rst b/docs/user_guide/bots/telegram_bot.rst
@@ -0,0 +1,6 @@
+Telegram bot
+===========
+
+.. automodule:: neptunecontrib.bots.telegram_bot
+    :members:
+    :show-inheritance:
diff --git a/docs/user_guide/sync/with_json.rst b/docs/user_guide/sync/with_json.rst
@@ -0,0 +1,6 @@
+Sync experiments with Neptune via json file
+===========
+
+.. automodule:: neptunecontrib.sync.with_json
+    :members:
+    :show-inheritance:
diff --git a/neptunecontrib/bots/__init__.py b/neptunecontrib/bots/__init__.py
@@ -0,0 +1,15 @@
+#
+# Copyright (c) 2019, Neptune Labs Sp. z o.o.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#