Skip to content

Update estimator examples #283

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
May 17, 2018
Merged

Update estimator examples #283

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
3c8b44d
unit tests pep8
leewyang May 16, 2018
31d6464
add tensorboard arg to estimator example; add keras/estimator example
leewyang May 16, 2018
092469f
more pep8; py2 vs. py3 compat
leewyang May 17, 2018
062057d
bump version
leewyang May 17, 2018
525f85b
use latest TF java API
leewyang May 17, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/.buildinfo
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Sphinx build info version 1
# This file hashes the configuration used when building these files. When it is not found, a full rebuild will be done.
config: c31f3f4c132de601af2c7dd3d80ea76f
config: 7466c912b353c5a104638dccc44e9814
tags: 645f666f9bcd5a90fca523b33c5a78b7
8 changes: 4 additions & 4 deletions docs/_modules/index.html
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
<head>
<meta http-equiv="X-UA-Compatible" content="IE=Edge" />
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<title>Overview: module code &#8212; TensorFlowOnSpark 1.3.0 documentation</title>
<title>Overview: module code &#8212; TensorFlowOnSpark 1.3.1 documentation</title>
<link rel="stylesheet" href="../_static/classic.css" type="text/css" />
<link rel="stylesheet" href="../_static/pygments.css" type="text/css" />
<script type="text/javascript" src="../_static/documentation_options.js"></script>
Expand All @@ -25,7 +25,7 @@ <h3>Navigation</h3>
<li class="right" >
<a href="../py-modindex.html" title="Python Module Index"
>modules</a> |</li>
<li class="nav-item nav-item-0"><a href="../index.html">TensorFlowOnSpark 1.3.0 documentation</a> &#187;</li>
<li class="nav-item nav-item-0"><a href="../index.html">TensorFlowOnSpark 1.3.1 documentation</a> &#187;</li>
</ul>
</div>

Expand Down Expand Up @@ -78,11 +78,11 @@ <h3>Navigation</h3>
<li class="right" >
<a href="../py-modindex.html" title="Python Module Index"
>modules</a> |</li>
<li class="nav-item nav-item-0"><a href="../index.html">TensorFlowOnSpark 1.3.0 documentation</a> &#187;</li>
<li class="nav-item nav-item-0"><a href="../index.html">TensorFlowOnSpark 1.3.1 documentation</a> &#187;</li>
</ul>
</div>
<div class="footer" role="contentinfo">
&#169; Copyright 2017, Yahoo Inc.
&#169; Copyright 2018, Yahoo Inc.
Created using <a href="http://sphinx-doc.org/">Sphinx</a> 1.7.1.
</div>
</body>
Expand Down
8 changes: 4 additions & 4 deletions docs/_modules/tensorflowonspark/TFCluster.html
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
<head>
<meta http-equiv="X-UA-Compatible" content="IE=Edge" />
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<title>tensorflowonspark.TFCluster &#8212; TensorFlowOnSpark 1.3.0 documentation</title>
<title>tensorflowonspark.TFCluster &#8212; TensorFlowOnSpark 1.3.1 documentation</title>
<link rel="stylesheet" href="../../_static/classic.css" type="text/css" />
<link rel="stylesheet" href="../../_static/pygments.css" type="text/css" />
<script type="text/javascript" src="../../_static/documentation_options.js"></script>
Expand All @@ -25,7 +25,7 @@ <h3>Navigation</h3>
<li class="right" >
<a href="../../py-modindex.html" title="Python Module Index"
>modules</a> |</li>
<li class="nav-item nav-item-0"><a href="../../index.html">TensorFlowOnSpark 1.3.0 documentation</a> &#187;</li>
<li class="nav-item nav-item-0"><a href="../../index.html">TensorFlowOnSpark 1.3.1 documentation</a> &#187;</li>
<li class="nav-item nav-item-1"><a href="../index.html" accesskey="U">Module code</a> &#187;</li>
</ul>
</div>
Expand Down Expand Up @@ -416,12 +416,12 @@ <h3>Navigation</h3>
<li class="right" >
<a href="../../py-modindex.html" title="Python Module Index"
>modules</a> |</li>
<li class="nav-item nav-item-0"><a href="../../index.html">TensorFlowOnSpark 1.3.0 documentation</a> &#187;</li>
<li class="nav-item nav-item-0"><a href="../../index.html">TensorFlowOnSpark 1.3.1 documentation</a> &#187;</li>
<li class="nav-item nav-item-1"><a href="../index.html" >Module code</a> &#187;</li>
</ul>
</div>
<div class="footer" role="contentinfo">
&#169; Copyright 2017, Yahoo Inc.
&#169; Copyright 2018, Yahoo Inc.
Created using <a href="http://sphinx-doc.org/">Sphinx</a> 1.7.1.
</div>
</body>
Expand Down
17 changes: 11 additions & 6 deletions docs/_modules/tensorflowonspark/TFManager.html
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
<head>
<meta http-equiv="X-UA-Compatible" content="IE=Edge" />
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<title>tensorflowonspark.TFManager &#8212; TensorFlowOnSpark 1.3.0 documentation</title>
<title>tensorflowonspark.TFManager &#8212; TensorFlowOnSpark 1.3.1 documentation</title>
<link rel="stylesheet" href="../../_static/classic.css" type="text/css" />
<link rel="stylesheet" href="../../_static/pygments.css" type="text/css" />
<script type="text/javascript" src="../../_static/documentation_options.js"></script>
Expand All @@ -25,7 +25,7 @@ <h3>Navigation</h3>
<li class="right" >
<a href="../../py-modindex.html" title="Python Module Index"
>modules</a> |</li>
<li class="nav-item nav-item-0"><a href="../../index.html">TensorFlowOnSpark 1.3.0 documentation</a> &#187;</li>
<li class="nav-item nav-item-0"><a href="../../index.html">TensorFlowOnSpark 1.3.1 documentation</a> &#187;</li>
<li class="nav-item nav-item-1"><a href="../index.html" accesskey="U">Module code</a> &#187;</li>
</ul>
</div>
Expand All @@ -48,6 +48,7 @@ <h1>Source code for tensorflowonspark.TFManager</h1><div class="highlight"><pre>
<span class="kn">from</span> <span class="nn">multiprocessing.managers</span> <span class="k">import</span> <span class="n">BaseManager</span>
<span class="kn">from</span> <span class="nn">multiprocessing</span> <span class="k">import</span> <span class="n">JoinableQueue</span>


<div class="viewcode-block" id="TFManager"><a class="viewcode-back" href="../../tensorflowonspark.TFManager.html#tensorflowonspark.TFManager.TFManager">[docs]</a><span class="k">class</span> <span class="nc">TFManager</span><span class="p">(</span><span class="n">BaseManager</span><span class="p">):</span>
<span class="sd">&quot;&quot;&quot;Python multiprocessing.Manager for distributed, multi-process communication.&quot;&quot;&quot;</span></div>
<span class="k">pass</span>
Expand All @@ -58,18 +59,22 @@ <h1>Source code for tensorflowonspark.TFManager</h1><div class="highlight"><pre>
<span class="n">qdict</span> <span class="o">=</span> <span class="p">{}</span> <span class="c1"># dictionary of queues</span>
<span class="n">kdict</span> <span class="o">=</span> <span class="p">{}</span> <span class="c1"># dictionary of key-values</span>


<span class="k">def</span> <span class="nf">_get</span><span class="p">(</span><span class="n">key</span><span class="p">):</span>
<span class="k">return</span> <span class="n">kdict</span><span class="p">[</span><span class="n">key</span><span class="p">]</span>


<span class="k">def</span> <span class="nf">_set</span><span class="p">(</span><span class="n">key</span><span class="p">,</span> <span class="n">value</span><span class="p">):</span>
<span class="n">kdict</span><span class="p">[</span><span class="n">key</span><span class="p">]</span> <span class="o">=</span> <span class="n">value</span>


<span class="k">def</span> <span class="nf">_get_queue</span><span class="p">(</span><span class="n">qname</span><span class="p">):</span>
<span class="k">try</span><span class="p">:</span>
<span class="k">return</span> <span class="n">qdict</span><span class="p">[</span><span class="n">qname</span><span class="p">]</span>
<span class="k">except</span> <span class="ne">KeyError</span><span class="p">:</span>
<span class="k">return</span> <span class="kc">None</span>


<div class="viewcode-block" id="start"><a class="viewcode-back" href="../../tensorflowonspark.TFManager.html#tensorflowonspark.TFManager.start">[docs]</a><span class="k">def</span> <span class="nf">start</span><span class="p">(</span><span class="n">authkey</span><span class="p">,</span> <span class="n">queues</span><span class="p">,</span> <span class="n">mode</span><span class="o">=</span><span class="s1">&#39;local&#39;</span><span class="p">):</span>
<span class="sd">&quot;&quot;&quot;Create a new multiprocess.Manager (or return existing one).</span>

Expand All @@ -91,12 +96,13 @@ <h1>Source code for tensorflowonspark.TFManager</h1><div class="highlight"><pre>
<span class="n">TFManager</span><span class="o">.</span><span class="n">register</span><span class="p">(</span><span class="s1">&#39;get&#39;</span><span class="p">,</span> <span class="n">callable</span><span class="o">=</span><span class="k">lambda</span> <span class="n">key</span><span class="p">:</span> <span class="n">_get</span><span class="p">(</span><span class="n">key</span><span class="p">))</span>
<span class="n">TFManager</span><span class="o">.</span><span class="n">register</span><span class="p">(</span><span class="s1">&#39;set&#39;</span><span class="p">,</span> <span class="n">callable</span><span class="o">=</span><span class="k">lambda</span> <span class="n">key</span><span class="p">,</span> <span class="n">value</span><span class="p">:</span> <span class="n">_set</span><span class="p">(</span><span class="n">key</span><span class="p">,</span> <span class="n">value</span><span class="p">))</span>
<span class="k">if</span> <span class="n">mode</span> <span class="o">==</span> <span class="s1">&#39;remote&#39;</span><span class="p">:</span>
<span class="n">mgr</span> <span class="o">=</span> <span class="n">TFManager</span><span class="p">(</span><span class="n">address</span><span class="o">=</span><span class="p">(</span><span class="s1">&#39;&#39;</span><span class="p">,</span><span class="mi">0</span><span class="p">),</span> <span class="n">authkey</span><span class="o">=</span><span class="n">authkey</span><span class="p">)</span>
<span class="n">mgr</span> <span class="o">=</span> <span class="n">TFManager</span><span class="p">(</span><span class="n">address</span><span class="o">=</span><span class="p">(</span><span class="s1">&#39;&#39;</span><span class="p">,</span> <span class="mi">0</span><span class="p">),</span> <span class="n">authkey</span><span class="o">=</span><span class="n">authkey</span><span class="p">)</span>
<span class="k">else</span><span class="p">:</span>
<span class="n">mgr</span> <span class="o">=</span> <span class="n">TFManager</span><span class="p">(</span><span class="n">authkey</span><span class="o">=</span><span class="n">authkey</span><span class="p">)</span>
<span class="n">mgr</span><span class="o">.</span><span class="n">start</span><span class="p">()</span></div>
<span class="k">return</span> <span class="n">mgr</span>


<div class="viewcode-block" id="connect"><a class="viewcode-back" href="../../tensorflowonspark.TFManager.html#tensorflowonspark.TFManager.connect">[docs]</a><span class="k">def</span> <span class="nf">connect</span><span class="p">(</span><span class="n">address</span><span class="p">,</span> <span class="n">authkey</span><span class="p">):</span>
<span class="sd">&quot;&quot;&quot;Connect to a multiprocess.Manager.</span>

Expand All @@ -113,7 +119,6 @@ <h1>Source code for tensorflowonspark.TFManager</h1><div class="highlight"><pre>
<span class="n">m</span> <span class="o">=</span> <span class="n">TFManager</span><span class="p">(</span><span class="n">address</span><span class="p">,</span> <span class="n">authkey</span><span class="o">=</span><span class="n">authkey</span><span class="p">)</span>
<span class="n">m</span><span class="o">.</span><span class="n">connect</span><span class="p">()</span></div>
<span class="k">return</span> <span class="n">m</span>

</pre></div>

</div>
Expand Down Expand Up @@ -146,12 +151,12 @@ <h3>Navigation</h3>
<li class="right" >
<a href="../../py-modindex.html" title="Python Module Index"
>modules</a> |</li>
<li class="nav-item nav-item-0"><a href="../../index.html">TensorFlowOnSpark 1.3.0 documentation</a> &#187;</li>
<li class="nav-item nav-item-0"><a href="../../index.html">TensorFlowOnSpark 1.3.1 documentation</a> &#187;</li>
<li class="nav-item nav-item-1"><a href="../index.html" >Module code</a> &#187;</li>
</ul>
</div>
<div class="footer" role="contentinfo">
&#169; Copyright 2017, Yahoo Inc.
&#169; Copyright 2018, Yahoo Inc.
Created using <a href="http://sphinx-doc.org/">Sphinx</a> 1.7.1.
</div>
</body>
Expand Down
10 changes: 5 additions & 5 deletions docs/_modules/tensorflowonspark/TFNode.html
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
<head>
<meta http-equiv="X-UA-Compatible" content="IE=Edge" />
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<title>tensorflowonspark.TFNode &#8212; TensorFlowOnSpark 1.3.0 documentation</title>
<title>tensorflowonspark.TFNode &#8212; TensorFlowOnSpark 1.3.1 documentation</title>
<link rel="stylesheet" href="../../_static/classic.css" type="text/css" />
<link rel="stylesheet" href="../../_static/pygments.css" type="text/css" />
<script type="text/javascript" src="../../_static/documentation_options.js"></script>
Expand All @@ -25,7 +25,7 @@ <h3>Navigation</h3>
<li class="right" >
<a href="../../py-modindex.html" title="Python Module Index"
>modules</a> |</li>
<li class="nav-item nav-item-0"><a href="../../index.html">TensorFlowOnSpark 1.3.0 documentation</a> &#187;</li>
<li class="nav-item nav-item-0"><a href="../../index.html">TensorFlowOnSpark 1.3.1 documentation</a> &#187;</li>
<li class="nav-item nav-item-1"><a href="../index.html" accesskey="U">Module code</a> &#187;</li>
</ul>
</div>
Expand Down Expand Up @@ -108,7 +108,7 @@ <h1>Source code for tensorflowonspark.TFNode</h1><div class="highlight"><pre>
<span class="n">cluster_spec</span> <span class="o">=</span> <span class="n">ctx</span><span class="o">.</span><span class="n">cluster_spec</span>
<span class="n">logging</span><span class="o">.</span><span class="n">info</span><span class="p">(</span><span class="s2">&quot;</span><span class="si">{0}</span><span class="s2">: Cluster spec: </span><span class="si">{1}</span><span class="s2">&quot;</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">ctx</span><span class="o">.</span><span class="n">worker_num</span><span class="p">,</span> <span class="n">cluster_spec</span><span class="p">))</span>

<span class="k">if</span> <span class="n">tf</span><span class="o">.</span><span class="n">test</span><span class="o">.</span><span class="n">is_built_with_cuda</span><span class="p">():</span>
<span class="k">if</span> <span class="n">tf</span><span class="o">.</span><span class="n">test</span><span class="o">.</span><span class="n">is_built_with_cuda</span><span class="p">()</span> <span class="ow">and</span> <span class="n">num_gpus</span> <span class="o">&gt;</span> <span class="mi">0</span><span class="p">:</span>
<span class="c1"># GPU</span>
<span class="n">gpu_initialized</span> <span class="o">=</span> <span class="kc">False</span>
<span class="n">retries</span> <span class="o">=</span> <span class="mi">3</span>
Expand Down Expand Up @@ -358,12 +358,12 @@ <h3>Navigation</h3>
<li class="right" >
<a href="../../py-modindex.html" title="Python Module Index"
>modules</a> |</li>
<li class="nav-item nav-item-0"><a href="../../index.html">TensorFlowOnSpark 1.3.0 documentation</a> &#187;</li>
<li class="nav-item nav-item-0"><a href="../../index.html">TensorFlowOnSpark 1.3.1 documentation</a> &#187;</li>
<li class="nav-item nav-item-1"><a href="../index.html" >Module code</a> &#187;</li>
</ul>
</div>
<div class="footer" role="contentinfo">
&#169; Copyright 2017, Yahoo Inc.
&#169; Copyright 2018, Yahoo Inc.
Created using <a href="http://sphinx-doc.org/">Sphinx</a> 1.7.1.
</div>
</body>
Expand Down
8 changes: 4 additions & 4 deletions docs/_modules/tensorflowonspark/TFSparkNode.html
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
<head>
<meta http-equiv="X-UA-Compatible" content="IE=Edge" />
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<title>tensorflowonspark.TFSparkNode &#8212; TensorFlowOnSpark 1.3.0 documentation</title>
<title>tensorflowonspark.TFSparkNode &#8212; TensorFlowOnSpark 1.3.1 documentation</title>
<link rel="stylesheet" href="../../_static/classic.css" type="text/css" />
<link rel="stylesheet" href="../../_static/pygments.css" type="text/css" />
<script type="text/javascript" src="../../_static/documentation_options.js"></script>
Expand All @@ -25,7 +25,7 @@ <h3>Navigation</h3>
<li class="right" >
<a href="../../py-modindex.html" title="Python Module Index"
>modules</a> |</li>
<li class="nav-item nav-item-0"><a href="../../index.html">TensorFlowOnSpark 1.3.0 documentation</a> &#187;</li>
<li class="nav-item nav-item-0"><a href="../../index.html">TensorFlowOnSpark 1.3.1 documentation</a> &#187;</li>
<li class="nav-item nav-item-1"><a href="../index.html" accesskey="U">Module code</a> &#187;</li>
</ul>
</div>
Expand Down Expand Up @@ -593,12 +593,12 @@ <h3>Navigation</h3>
<li class="right" >
<a href="../../py-modindex.html" title="Python Module Index"
>modules</a> |</li>
<li class="nav-item nav-item-0"><a href="../../index.html">TensorFlowOnSpark 1.3.0 documentation</a> &#187;</li>
<li class="nav-item nav-item-0"><a href="../../index.html">TensorFlowOnSpark 1.3.1 documentation</a> &#187;</li>
<li class="nav-item nav-item-1"><a href="../index.html" >Module code</a> &#187;</li>
</ul>
</div>
<div class="footer" role="contentinfo">
&#169; Copyright 2017, Yahoo Inc.
&#169; Copyright 2018, Yahoo Inc.
Created using <a href="http://sphinx-doc.org/">Sphinx</a> 1.7.1.
</div>
</body>
Expand Down
Loading