Tensorflow support for DIGITS #1714

ethantang95 · 2017-07-06T18:05:38Z

@gheinrich @TimZaman @lukeyeager for review please

TimZaman

I think we need to be pragmatic with this review. I've already been through most of this several times. I think we need to be pragmatic here. I'm going to:
(1) again, go through all the docs as a user one final time.
(2) start install from 0 and see if everything works for me

TimZaman · 2017-07-09T13:52:15Z

digits/frameworks/framework.py

+        """
+        return self.SUPPORTS_PYTHON_LAYERS_FILE
+
+    def supports_timeline_traces(self):


Ah yeah, timeline traces. Those were pretty fancy. Did you ever test this? You don't need to.

TimZaman · 2017-07-09T13:55:00Z

digits/model/images/generic/test_views.py

+        super(TestTensorflowCreatedWithGradientDataExtensionNoValSet, cls).setUpClass(val_image_count=0)
+
+
+# class TestTensorflowCreatedWithImageProcessingExtensionMeanImage(BaseTestCreatedWithImageProcessingExtension,


What's the story here? these didnt work?

I never noticed this was commented out. I'll retest it

Looks to be working fine

TimZaman · 2017-07-09T14:04:49Z

I just verified the CI job here just ran only 14 tests for tensorflow, while it ran 437 for torch and 388 for caffe. I thought you fixed this? This needs fixing asap!
We can run the specific tensorflow tests now locally using:

export DIGITS_TEST_FRAMEWORK=tensorflow
./digits-test -v

Running them now again on my @desktop, looks good, thankfully. But CI really needs to be working though! I ran 247 tensorflow tests successfully. Phew!

TimZaman · 2017-07-09T14:09:46Z

You also forgot to change to update the versions from 5.x to 6.0! I already noted this in a previous review.. See for example DIGITS/digits/version.py.

TimZaman · 2017-07-09T14:11:55Z

digits/dataset/images/classification/forms.py

@@ -21,13 +21,16 @@ class ImageClassificationDatasetForm(ImageDatasetForm):
                                  choices=[
                                      ('lmdb', 'LMDB'),
                                      ('hdf5', 'HDF5'),
+                                      ('tfrecords', 'TFRecords'),


Hey, come on, it seems you've adopted nothing from our previous reviews. This should be removed, so that we do have the back-end implementation for it, but people don't use it as it's currently not working as it should.

You said previously to keep it in case some others want to extend it and get it working?

I am not in favor of merging a feature which isn't working.

The generator (of tfrecords) actually works fine. But I don't want to support it (for training). So block out the item so people can't click on it anymore.

TimZaman · 2017-07-09T14:20:31Z

MNIST HDF5 intestion works. MNIST LMDB ingestion works. Tensorflow MNIST-HDF5 training works. Tensorflow MNIST-LMDB training works.

gheinrich · 2017-07-10T13:37:39Z

Hi Ethan, not a huge deal but you have assumed authorship of all my changes. You can easily fixed this with a git commit --amend --author="...". If you do this, it's not quite necessary to name the author in the commit message as it's implicitly captured by Git.

TimZaman · 2017-07-10T13:39:56Z

Greg, do you have some time to check the functionality of tour GAN tutorial? Figuring it might be popular.

…

On Mon, 10 Jul 2017 at 15:37, Greg Heinrich ***@***.***> wrote: Hi Ethan, not a huge deal but you have assumed authorship of all my changes. You can easily fixed this with a git commit --amend --author="...". If you do this, it's not quite necessary to name the author in the commit message as it's implicitly captured by Git. — You are receiving this because you were assigned. Reply to this email directly, view it on GitHub <#1714 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AHXSRDwp3c9WPqhJk9d3IznZvhKQFWFOks5sMikygaJpZM4OP_MT> .

gheinrich

Hi Ethan, can you rebase the tensorflow branch? A lot of changes are already on master, which makes it difficult to figure out what you changed there v.s. what you got from master.

gheinrich · 2017-07-10T13:30:57Z

digits/config/store_option.py

@@ -29,7 +29,7 @@ def load_url_list():
    if 'DIGITS_MODEL_STORE_URL' in os.environ:
        url_list = os.environ['DIGITS_MODEL_STORE_URL']
    else:
-        url_list = ""
+        url_list = "http://developer.download.nvidia.com/compute/machine-learning/modelstore/5.0"


I thought you were planning to add the GAN model there. In which case you might want to move to a 6.0 version?

gheinrich · 2017-07-10T13:31:15Z

digits/config/tensorflow.py

@@ -34,7 +34,7 @@ def test_tf_import(python_exe):

 if not tf_enabled:
    print('Tensorflow support disabled.')
-#    print('Failed importing Tensorflow with python executable "%s"\n%s' % (tf_python_exe, err))
+#   print('Failed importing Tensorflow with python executable "%s"\n%s' % (tf_python_exe, err))


you could remove this altogether

for now, I need it to debug the travis-CI problem. Will remove in the merge to master

gheinrich · 2017-07-10T13:34:59Z

digits/dataset/images/classification/views.py

@@ -156,12 +157,14 @@ def from_files(job, form):
    """
    # labels
    if form.textfile_use_local_files.data:
-        job.labels_file = form.textfile_local_labels_file.data.strip()
+        labels_file_from = form.textfile_local_labels_file.data.strip()


This change is already on master. You might want to rebase the tensorflow branch into master first - then add your patches. This will keep commit history cleaner.

gheinrich · 2017-07-10T13:35:40Z

digits/dataset/tasks/create_db.py

@@ -100,6 +100,14 @@ def __setstate__(self, state):
        if not hasattr(self, 'compression') or self.compression is None:
            self.compression = 'none'

+        if not hasattr(self, 'entries_error'):


this change is also on master

gheinrich · 2017-07-10T13:36:20Z

digits/dataset/tasks/create_db.py

-                              namespace='/jobs',
-                              room=self.job_id,
-                              )
+            self.distribution[match.group(1)] = {


this change is also on master

gheinrich · 2017-07-10T13:41:41Z

digits/model/tasks/caffe_train.py

-                                                                   normalize=normalize,
-                                                                   allow_heatmap=bool(top != 'data'),
-                                                                   channel_order='BGR')
+                                vis = utils.image.get_layer_vis_square(data * 255,


this change is also on master.

gheinrich · 2017-07-10T13:45:10Z

Hi @ethantang95 I see you've merged your branch and tensorflow. For easier-to-read reviews, rebasing is preferred, so that we can see exactly which patches you're adding to the target branch.

gheinrich · 2017-07-10T13:45:35Z

@TimZaman yes I can double check GANs.

lukeyeager · 2017-07-11T00:23:27Z

Finally got the git history all cleaned up. It's backed up here. Reviewing now ...

lukeyeager

Pfew, this is a lot of code to review! Here's a start. I'll keep working through it tomorrow. NOTE: so far, I'm only looking at coding style and decisions. Haven't actually tested that anything runs.

lukeyeager · 2017-07-11T00:27:59Z

README.md

@@ -4,6 +4,8 @@

 DIGITS (the **D**eep Learning **G**PU **T**raining **S**ystem) is a webapp for training deep learning models.

+The currently supported frameworks are: Caffe 1, Torch, and Tensorflow
+


Remove the newline so this section is just one paragraph. Call it just "Caffe", not "Caffe 1". Add a period at the end.

lukeyeager · 2017-07-11T00:30:19Z

README.md

@@ -44,4 +47,3 @@ Then, take a look at some of the other documentation at [docs/](docs/) and [exam
 * Please let us know by [filing a new issue](https://github.com/NVIDIA/DIGITS/issues/new)
 * Bonus points if you want to contribute by opening a [pull request](https://help.github.com/articles/using-pull-requests/)!
  * You will need to send a signed copy of the [Contributor License Agreement](CLA) to digits@nvidia.com before your change can be accepted.
-


Unnecessary

lukeyeager · 2017-07-11T00:32:03Z

digits-lint

 else
-    python2 -m flake8 .
+    python2 -m flake8 --exclude ./examples,./digits/standard-networks/tensorflow,./digits/jobs .


Our examples and standard networks should typically be lint-free. Is there a good reason to exclude these? I'm OK with the digits/jobs exclusion.

The reason I excluded those is because of linting complaining about not being to find reference to UserModel or Tower or model_property in those files

Can we import them at the top of the file, then?

lukeyeager · 2017-07-11T00:33:35Z

digits/config/tensorflow.py

+from . import option_list
+
+VARNAME_ENV_TFPY = 'TENSORFLOW_PYTHON'
+DEFAULT_PYTHON_EXE = 'python2'  # @TODO(tzaman) - use the python executable that was used to launch digits?


@TimZaman we still want to do this, right?

@ethantang95 here's how we do it elsewhere:
https://github.com/NVIDIA/DIGITS/blob/v5.0.0/digits/dataset/tasks/analyze_db.py#L78

lukeyeager · 2017-07-11T00:34:33Z

digits/config/tensorflow.py

+    # DYLD_LIBRARY_PATH and LD_LIBRARY_PATH is sometimes stripped, and the cuda libraries might need it
+    if "DYLD_LIBRARY_PATH" not in os.environ:
+        if "CUDA_HOME" in os.environ:
+            os.environ["DYLD_LIBRARY_PATH"] = str(os.environ["CUDA_HOME"] + '/lib')


Why are we fixing people's bad environment for them here? Shouldn't this be a standard TF configuration issue?

it's a bit more complicated than that but this can be removed yeah

lukeyeager · 2017-07-11T00:48:36Z

digits/model/forms.py

+#        if form.method.data == 'custom':
+#            for filename in field.data.strip().split(os.path.pathsep):
+#                if filename and not os.path.exists(filename):
+#                    raise validators.ValidationError('File "%s" does not exist' % filename)


Uh, why would we remove this check?

Ethan, try to see if you can do a framework check here.

nothing references this method even

Just verified, you seem to be right. If it's invoked it's through flask?

yeah looks like it

So the problem is with tensorflow files where it has the .data-00000-of-00001 problem. Add a fix or leave it?

lukeyeager · 2017-07-11T00:50:46Z

digits/model/images/classification/test_views.py

-@unittest.skipIf(
-    not CaffeFramework().can_accumulate_gradients(),
-    'This version of Caffe cannot accumulate gradients')
-class TestBatchAccumulationCaffe(BaseViewsTestWithDataset, test_utils.CaffeMixin):


Why are we getting rid of this test?

@ethantang95 lets see if this can be re added

it passes. Add it back?

More tests is better, so yeah.

lukeyeager · 2017-07-11T00:52:31Z

digits/model/tasks/train.py

@@ -431,7 +435,7 @@ def infer_many(self, data, model_epoch=None):
        """
        return None

-    def get_snapshot(self, epoch=-1):
+    def get_snapshot(self, epoch=-1, download=False):


Why are we adding the download keyword here? It's not being used...

the flag was added here even though it's not used in order to keep a consistent interface between Tensorflow and other frameworks.

lukeyeager · 2017-07-11T00:55:03Z

digits/model/images/classification/views.py

-                                    break
-
+                            # verify snapshot exists
+                            pretrained_model = old_job.train_task().get_snapshot(epoch, download=True)


Why this change? It's better to throw a BadRequest than the ValueError that gets thrown if the epoch is missing unless there's a reason not to.

This is unclear even to me..

According to git history, @gheinrich was the one that added the line. @gheinrich can you comment on this please?

lukeyeager · 2017-07-11T00:56:33Z

digits/model/images/generic/views.py

-                                    break
-
+                            # verify snapshot exists
+                            pretrained_model = old_job.train_task().get_snapshot(epoch, download=True)


This download flag comes from the fact that Tensorflow appends the .data-00000-of-00001 suffix to the files it creates. So if you do something like tf.save('toto') it is going to create a file named toto.data-00000-of-00001. If you want to download the snapshot you need to get the file toto.data-00000-of-00001. The complication comes from the fact that if you want to reload the snapshot in Tensorflow you can't do tf.load('toto.data-00000-of-00001') you need to do tf.load('toto')...

I could have implemented this change differently and referred to the actual filename in get_snapshot() and I would just have had to strip the data-00000-of-00001 when reloading the snapshot in TF. That would have isolated the change to the TF implementation. I guess I didn't think it through thoroughly at the time. Didn't really expect to ever release this though :-)

Oh I see. Ok. Let's just leave it, then. Thanks for the explanation!

lukeyeager · 2017-07-11T01:02:32Z

I'd take another look at how Tensorflow is being tested on Travis:

Caffe: Ran 388 tests in 445.581s
Torch: Ran 457 tests in 512.154s
Tensorflow: Ran 14 tests in 1.752s

ethantang95 · 2017-07-11T03:47:30Z

I am aware of that problem and it is being fixed. However the tensorflow tests do pass perfectly locally without a problem. I just think it's a problem with the scripts that is causing that

lukeyeager

There's really no way to properly review a PR this huge. I wish we would have been able to do this with more, smaller PRs. Oh well. Here's some feedback.

lukeyeager · 2017-07-11T16:59:07Z

digits/model/tasks/tensorflow_train.py

+        if not snapshot_filename:
+            raise ValueError('Invalid epoch')
+        if download:
+            snapshot_filename = snapshot_filename + ".data-00000-of-00001"


Just for my own edification, what is this about?

I think it's because of the weird way tensorflow name snapshots... @gheinrich can you comment on this?

yes, annoyingly TensorFlow doesn't save snapshots using exactly the specified file name but happens this .data-00000-of-00001 (possibly to make sharding possible if needed).

Why would we ever want download=False? This little change seems to have spidered out into a surprisingly large number of files.

guess it is due to tensorflow's .data-00000-of-00001 problem. That is the time the variable is used.

lukeyeager · 2017-07-11T16:59:32Z

digits/model/tasks/tensorflow_train.py

+    @override
+    def task_arguments(self, resources, env):
+
+        args = [config_value('tensorflow')['executable'],


Why not sys.executable as with other tasks?

lukeyeager · 2017-07-11T17:50:07Z

digits/model/tasks/tensorflow_train.py

+    @staticmethod
+    def preprocess_output_tensorflow(line):
+        """
+        Takes line of output and parses it according to caffe's output format


"caffe" -> "tensorflow"

lukeyeager · 2017-07-11T17:58:34Z

digits/standard-networks/tensorflow/alexnet_slim.py

@@ -0,0 +1,38 @@
+from model import Tower


Why are we checking in two standard networks for AlexNet? Looks like only alexnet.py is used:
https://github.com/lukeyeager/DIGITS/blob/backup/ethan/tf-pr-20170710/digits/frameworks/tensorflow_framework.py#L63
Could we scrap the non-slim versions of these standard networks and just check-in the ones which use the slim API?

I think Tim wants to have both version to show that if someone want to do slim, they can... @TimZaman can you confirm?

They are two different approaches. One could even make a model with keras. It's up to the user. I don't mind much. What do you think @lukeyeager ? We can remove the slim versions?

I like the conciseness of the slim versions. Are they functionally equivalent?

yeah equivalent. one could even use keras. Tensorflow vanilla is really not concise, which is why keras is so popular :D

lukeyeager · 2017-07-11T18:02:32Z

digits/static/tb/tf-graph-basic.build.js

@@ -0,0 +1,20 @@
+!function(){window.WebComponents=window.WebComponents||{flags:{}};var a="webcomponents-lite.js",b=document.querySelector('script[src*="'+a+'"]'),c={};if(!c.noOpts){if(location.search.slice(1).split("&").forEach(function(a){var b,d=a.split("=");d[0]&&(b=d[0].match(/wc-(.+)/))&&(c[b[1]]=d[1]||!0)}),b)for(var d,e=0;d=b.attributes[e];e++)"src"!==d.name&&(c[d.name]=d.value||!0);if(c.log&&c.log.split){var f=c.log.split(",");c.log={},f.forEach(function(a){c.log[a]=!0})}else c.log={}}c.register&&(window.CustomElements=window.CustomElements||{flags:{}},window.CustomElements.flags.register=c.register),WebComponents.flags=c}(),function(a){"use strict";function b(a){return void 0!==l[a]}function c(){h.call(this),this._isInvalid=!0}function d(a){return""==a&&c.call(this),a.toLowerCase()}function e(a){var b=a.charCodeAt(0);return b>32&&127>b&&-1==[34,35,60,62,63,96].indexOf(b)?a:encodeURIComponent(a)}function f(a){var b=a.charCodeAt(0);return b>32&&127>b&&-1==[34,35,60,62,96].indexOf(b)?a:encodeURIComponent(a)}function g(a,g,h){function i(a){t.push(a)}var j=g||"scheme start",k=0,q="",r=!1,s=!1,t=[];a:for(;(a[k-1]!=n||0==k)&&!this._isInvalid;){var u=a[k];switch(j){case"scheme start":if(!u||!o.test(u)){if(g){i("Invalid scheme.");break a}q="",j="no scheme";continue}q+=u.toLowerCase(),j="scheme";break;case"scheme":if(u&&p.test(u))q+=u.toLowerCase();else{if(":"!=u){if(g){if(n==u)break a;i("Code point not allowed in scheme: "+u);break a}q="",k=0,j="no scheme";continue}if(this._scheme=q,q="",g)break a;b(this._scheme)&&(this._isRelative=!0),j="file"==this._scheme?"relative":this._isRelative&&h&&h._scheme==this._scheme?"relative or authority":this._isRelative?"authority first slash":"scheme data"}break;case"scheme data":"?"==u?(this._query="?",j="query"):"#"==u?(this._fragment="#",j="fragment"):n!=u&&"\t"!=u&&"\n"!=u&&"\r"!=u&&(this._schemeData+=e(u));break;case"no scheme":if(h&&b(h._scheme)){j="relative";continue}i("Missing scheme."),c.call(this);break;case"relative or authority":if("/"!=u||"/"!=a[k+1]){i("Expected /, got: "+u),j="relative";continue}j="authority ignore slashes";break;case"relative":if(this._isRelative=!0,"file"!=this._scheme&&(this._scheme=h._scheme),n==u){this._host=h._host,this._port=h._port,this._path=h._path.slice(),this._query=h._query,this._username=h._username,this._password=h._password;break a}if("/"==u||"\\"==u)"\\"==u&&i("\\ is an invalid code point."),j="relative slash";else if("?"==u)this._host=h._host,this._port=h._port,this._path=h._path.slice(),this._query="?",this._username=h._username,this._password=h._password,j="query";else{if("#"!=u){var v=a[k+1],w=a[k+2];("file"!=this._scheme||!o.test(u)||":"!=v&&"|"!=v||n!=w&&"/"!=w&&"\\"!=w&&"?"!=w&&"#"!=w)&&(this._host=h._host,this._port=h._port,this._username=h._username,this._password=h._password,this._path=h._path.slice(),this._path.pop()),j="relative path";continue}this._host=h._host,this._port=h._port,this._path=h._path.slice(),this._query=h._query,this._fragment="#",this._username=h._username,this._password=h._password,j="fragment"}break;case"relative slash":if("/"!=u&&"\\"!=u){"file"!=this._scheme&&(this._host=h._host,this._port=h._port,this._username=h._username,this._password=h._password),j="relative path";continue}"\\"==u&&i("\\ is an invalid code point."),j="file"==this._scheme?"file host":"authority ignore slashes";break;case"authority first slash":if("/"!=u){i("Expected '/', got: "+u),j="authority ignore slashes";continue}j="authority second slash";break;case"authority second slash":if(j="authority ignore slashes","/"!=u){i("Expected '/', got: "+u);continue}break;case"authority ignore slashes":if("/"!=u&&"\\"!=u){j="authority";continue}i("Expected authority, got: "+u);break;case"authority":if("@"==u){r&&(i("@ already seen."),q+="%40"),r=!0;for(var x=0;x<q.length;x++){var y=q[x];if("\t"!=y&&"\n"!=y&&"\r"!=y)if(":"!=y||null!==this._password){var z=e(y);null!==this._password?this._password+=z:this._username+=z}else this._password="";else i("Invalid whitespace in authority.")}q=""}else{if(n==u||"/"==u||"\\"==u||"?"==u||"#"==u){k-=q.length,q="",j="host";continue}q+=u}break;case"file host":if(n==u||"/"==u||"\\"==u||"?"==u||"#"==u){2!=q.length||!o.test(q[0])||":"!=q[1]&&"|"!=q[1]?0==q.length?j="relative path start":(this._host=d.call(this,q),q="",j="relative path start"):j="relative path";continue}"\t"==u||"\n"==u||"\r"==u?i("Invalid whitespace in file host."):q+=u;break;case"host":case"hostname":if(":"!=u||s){if(n==u||"/"==u||"\\"==u||"?"==u||"#"==u){if(this._host=d.call(this,q),q="",j="relative path start",g)break a;continue}"\t"!=u&&"\n"!=u&&"\r"!=u?("["==u?s=!0:"]"==u&&(s=!1),q+=u):i("Invalid code point in host/hostname: "+u)}else if(this._host=d.call(this,q),q="",j="port","hostname"==g)break a;break;case"port":if(/[0-9]/.test(u))q+=u;else{if(n==u||"/"==u||"\\"==u||"?"==u||"#"==u||g){if(""!=q){var A=parseInt(q,10);A!=l[this._scheme]&&(this._port=A+""),q=""}if(g)break a;j="relative path start";continue}"\t"==u||"\n"==u||"\r"==u?i("Invalid code point in port: "+u):c.call(this)}break;case"relative path start":if("\\"==u&&i("'\\' not allowed in path."),j="relative path","/"!=u&&"\\"!=u)continue;break;case"relative path":if(n!=u&&"/"!=u&&"\\"!=u&&(g||"?"!=u&&"#"!=u))"\t"!=u&&"\n"!=u&&"\r"!=u&&(q+=e(u));else{"\\"==u&&i("\\ not allowed in relative path.");var B;(B=m[q.toLowerCase()])&&(q=B),".."==q?(this._path.pop(),"/"!=u&&"\\"!=u&&this._path.push("")):"."==q&&"/"!=u&&"\\"!=u?this._path.push(""):"."!=q&&("file"==this._scheme&&0==this._path.length&&2==q.length&&o.test(q[0])&&"|"==q[1]&&(q=q[0]+":"),this._path.push(q)),q="","?"==u?(this._query="?",j="query"):"#"==u&&(this._fragment="#",j="fragment")}break;case"query":g||"#"!=u?n!=u&&"\t"!=u&&"\n"!=u&&"\r"!=u&&(this._query+=f(u)):(this._fragment="#",j="fragment");break;case"fragment":n!=u&&"\t"!=u&&"\n"!=u&&"\r"!=u&&(this._fragment+=u)}k++}}function h(){this._scheme="",this._schemeData="",this._username="",this._password=null,this._host="",this._port="",this._path=[],this._query="",this._fragment="",this._isInvalid=!1,this._isRelative=!1}function i(a,b){void 0===b||b instanceof i||(b=new i(String(b))),this._url=a,h.call(this);var c=a.replace(/^[ \t\r\n\f]+|[ \t\r\n\f]+$/g,"");g.call(this,c,null,b)}var j=!1;if(!a.forceJURL)try{var k=new URL("b","http://a");k.pathname="c%20d",j="http://a/c%20d"===k.href}catch(a){}if(!j){var l=Object.create(null);l.ftp=21,l.file=0,l.gopher=70,l.http=80,l.https=443,l.ws=80,l.wss=443;var m=Object.create(null);m["%2e"]=".",m[".%2e"]="..",m["%2e."]="..",m["%2e%2e"]="..";var n=void 0,o=/[a-zA-Z]/,p=/[a-zA-Z0-9\+\-\.]/;i.prototype={toString:function(){return this.href},get href(){if(this._isInvalid)return this._url;var a="";return(""!=this._username||null!=this._password)&&(a=this._username+(null!=this._password?":"+this._password:"")+"@"),this.protocol+(this._isRelative?"//"+a+this.host:"")+this.pathname+this._query+this._fragment},set href(a){h.call(this),g.call(this,a)},get protocol(){return this._scheme+":"},set protocol(a){this._isInvalid||g.call(this,a+":","scheme start")},get host(){return this._isInvalid?"":this._port?this._host+":"+this._port:this._host},set host(a){!this._isInvalid&&this._isRelative&&g.call(this,a,"host")},get hostname(){return this._host},set hostname(a){!this._isInvalid&&this._isRelative&&g.call(this,a,"hostname")},get port(){return this._port},set port(a){!this._isInvalid&&this._isRelative&&g.call(this,a,"port")},get pathname(){return this._isInvalid?"":this._isRelative?"/"+this._path.join("/"):this._schemeData},set pathname(a){!this._isInvalid&&this._isRelative&&(this._path=[],g.call(this,a,"relative path start"))},get search(){return this._isInvalid||!this._query||"?"==this._query?"":this._query},set search(a){!this._isInvalid&&this._isRelative&&(this._query="?","?"==a[0]&&(a=a.slice(1)),g.call(this,a,"query"))},get hash(){return this._isInvalid||!this._fragment||"#"==this._fragment?"":this._fragment},set hash(a){this._isInvalid||(this._fragment="#","#"==a[0]&&(a=a.slice(1)),g.call(this,a,"fragment"))},get origin(){var a;if(this._isInvalid||!this._scheme)return"";switch(this._scheme){case"data":case"file":case"javascript":case"mailto":return"null"}return a=this.host,a?this._scheme+"://"+a:""}};var q=a.URL;q&&(i.createObjectURL=function(a){return q.createObjectURL.apply(q,arguments)},i.revokeObjectURL=function(a){q.revokeObjectURL(a)}),a.URL=i}}(self),"undefined"==typeof WeakMap&&!function(){var a=Object.defineProperty,b=Date.now()%1e9,c=function(){this.name="__st"+(1e9*Math.random()>>>0)+(b++ +"__")};c.prototype={set:function(b,c){var d=b[this.name];return d&&d[0]===b?d[1]=c:a(b,this.name,{value:[b,c],writable:!0}),this},get:function(a){var b;return(b=a[this.name])&&b[0]===a?b[1]:void 0},delete:function(a){var b=a[this.name];return!(!b||b[0]!==a)&&(b[0]=b[1]=void 0,!0)},has:function(a){var b=a[this.name];return!!b&&b[0]===a}},window.WeakMap=c}(),function(a){function b(a){u.push(a),t||(t=!0,p(d))}function c(a){return window.ShadowDOMPolyfill&&window.ShadowDOMPolyfill.wrapIfNeeded(a)||a}function d(){t=!1;var a=u;u=[],a.sort(function(a,b){return a.uid_-b.uid_});var b=!1;a.forEach(function(a){var c=a.takeRecords();e(a),c.length&&(a.callback_(c,a),b=!0)}),b&&d()}function e(a){a.nodes_.forEach(function(b){var c=q.get(b);c&&c.forEach(function(b){b.observer===a&&b.removeTransientObservers()})})}function f(a,b){for(var c=a;c;c=c.parentNode){var d=q.get(c);if(d)for(var e=0;e<d.length;e++){var f=d[e],g=f.options;if(c===a||g.subtree){var h=b(g);h&&f.enqueue(h)}}}}function g(a){this.callback_=a,this.nodes_=[],this.records_=[],this.uid_=++v}function h(a,b){this.type=a,this.target=b,this.addedNodes=[],this.removedNodes=[],this.previousSibling=null,this.nextSibling=null,this.attributeName=null,this.attributeNamespace=null,this.oldValue=null}function i(a){var b=new h(a.type,a.target);return b.addedNodes=a.addedNodes.slice(),b.removedNodes=a.removedNodes.slice(),b.previousSibling=a.previousSibling,b.nextSibling=a.nextSibling,b.attributeName=a.attributeName,b.attributeNamespace=a.attributeNamespace,b.oldValue=a.oldValue,b}function j(a,b){return w=new h(a,b)}function k(a){return x?x:(x=i(w),x.oldValue=a,x)}function l(){w=x=void 0}function m(a){return a===x||a===w}function n(a,b){return a===b?a:x&&m(a)?x:null}function o(a,b,c){this.observer=a,this.target=b,this.options=c,this.transientObservedNodes=[]}if(!a.JsMutationObserver){var p,q=new WeakMap;if(/Trident|Edge/.test(navigator.userAgent))p=setTimeout;else if(window.setImmediate)p=window.setImmediate;else{var r=[],s=String(Math.random());window.addEventListener("message",function(a){if(a.data===s){var b=r;r=[],b.forEach(function(a){a()})}}),p=function(a){r.push(a),window.postMessage(s,"*")}}var t=!1,u=[],v=0;g.prototype={observe:function(a,b){if(a=c(a),!b.childList&&!b.attributes&&!b.characterData||b.attributeOldValue&&!b.attributes||b.attributeFilter&&b.attributeFilter.length&&!b.attributes||b.characterDataOldValue&&!b.characterData)throw new SyntaxError;var d=q.get(a);d||q.set(a,d=[]);for(var e,f=0;f<d.length;f++)if(d[f].observer===this){e=d[f],e.removeListeners(),e.options=b;break}e||(e=new o(this,a,b),d.push(e),this.nodes_.push(a)),e.addListeners()},disconnect:function(){this.nodes_.forEach(function(a){for(var b=q.get(a),c=0;c<b.length;c++){var d=b[c];if(d.observer===this){d.removeListeners(),b.splice(c,1);break}}},this),this.records_=[]},takeRecords:function(){var a=this.records_;return this.records_=[],a}};var w,x;o.prototype={enqueue:function(a){var c=this.observer.records_,d=c.length;if(c.length>0){var e=c[d-1],f=n(e,a);if(f)return void(c[d-1]=f)}else b(this.observer);c[d]=a},addListeners:function(){this.addListeners_(this.target)},addListeners_:function(a){var b=this.options;b.attributes&&a.addEventListener("DOMAttrModified",this,!0),b.characterData&&a.addEventListener("DOMCharacterDataModified",this,!0),b.childList&&a.addEventListener("DOMNodeInserted",this,!0),(b.childList||b.subtree)&&a.addEventListener("DOMNodeRemoved",this,!0)},removeListeners:function(){this.removeListeners_(this.target)},removeListeners_:function(a){var b=this.options;b.attributes&&a.removeEventListener("DOMAttrModified",this,!0),b.characterData&&a.removeEventListener("DOMCharacterDataModified",this,!0),b.childList&&a.removeEventListener("DOMNodeInserted",this,!0),(b.childList||b.subtree)&&a.removeEventListener("DOMNodeRemoved",this,!0)},addTransientObserver:function(a){if(a!==this.target){this.addListeners_(a),this.transientObservedNodes.push(a);var b=q.get(a);b||q.set(a,b=[]),b.push(this)}},removeTransientObservers:function(){var a=this.transientObservedNodes;this.transientObservedNodes=[],a.forEach(function(a){this.removeListeners_(a);for(var b=q.get(a),c=0;c<b.length;c++)if(b[c]===this){b.splice(c,1);break}},this)},handleEvent:function(a){switch(a.stopImmediatePropagation(),a.type){case"DOMAttrModified":var b=a.attrName,c=a.relatedNode.namespaceURI,d=a.target,e=new j("attributes",d);e.attributeName=b,e.attributeNamespace=c;var g=a.attrChange===MutationEvent.ADDITION?null:a.prevValue;f(d,function(a){return!a.attributes||a.attributeFilter&&a.attributeFilter.length&&-1===a.attributeFilter.indexOf(b)&&-1===a.attributeFilter.indexOf(c)?void 0:a.attributeOldValue?k(g):e});break;case"DOMCharacterDataModified":var d=a.target,e=j("characterData",d),g=a.prevValue;f(d,function(a){return a.characterData?a.characterDataOldValue?k(g):e:void 0});break;case"DOMNodeRemoved":this.addTransientObserver(a.target);case"DOMNodeInserted":var h,i,m=a.target;"DOMNodeInserted"===a.type?(h=[m],i=[]):(h=[],i=[m]);var n=m.previousSibling,o=m.nextSibling,e=j("childList",a.target.parentNode);e.addedNodes=h,e.removedNodes=i,e.previousSibling=n,e.nextSibling=o,f(a.relatedNode,function(a){return a.childList?e:void 0})}l()}},a.JsMutationObserver=g,a.MutationObserver||(a.MutationObserver=g,g._isPolyfilled=!0)}}(self),"undefined"==typeof HTMLTemplateElement&&!function(){function a(a){switch(a){case"&":return"&amp;";case"<":return"&lt;";case">":return"&gt;";case" ":return"&nbsp;"}}function b(b){return b.replace(g,a)}var c="template",d=document.implementation.createHTMLDocument("template"),e=!0;HTMLTemplateElement=function(){},HTMLTemplateElement.prototype=Object.create(HTMLElement.prototype),HTMLTemplateElement.decorate=function(a){a.content||(a.content=d.createDocumentFragment());for(var c;c=a.firstChild;)a.content.appendChild(c);if(e)try{Object.defineProperty(a,"innerHTML",{get:function(){for(var a="",c=this.content.firstChild;c;c=c.nextSibling)a+=c.outerHTML||b(c.data);return a},set:function(a){for(d.body.innerHTML=a,HTMLTemplateElement.bootstrap(d);this.content.firstChild;)this.content.removeChild(this.content.firstChild);for(;d.body.firstChild;)this.content.appendChild(d.body.firstChild)},configurable:!0})}catch(a){e=!1}},HTMLTemplateElement.bootstrap=function(a){for(var b,d=a.querySelectorAll(c),e=0,f=d.length;f>e&&(b=d[e]);e++)HTMLTemplateElement.decorate(b)},document.addEventListener("DOMContentLoaded",function(){HTMLTemplateElement.bootstrap(document)});var f=document.createElement;document.createElement=function(){"use strict";var a=f.apply(document,arguments);return"template"==a.localName&&HTMLTemplateElement.decorate(a),a};var g=/[&\u00A0<>]/g}(),function(a){"use strict";if(!window.performance){var b=Date.now();window.performance={now:function(){return Date.now()-b}}}window.requestAnimationFrame||(window.requestAnimationFrame=function(){var a=window.webkitRequestAnimationFrame||window.mozRequestAnimationFrame;return a?function(b){return a(function(){b(performance.now())})}:function(a){return window.setTimeout(a,1e3/60)}}()),window.cancelAnimationFrame||(window.cancelAnimationFrame=function(){return window.webkitCancelAnimationFrame||window.mozCancelAnimationFrame||function(a){clearTimeout(a)}}());var c=function(){var a=document.createEvent("Event");return a.initEvent("foo",!0,!0),a.preventDefault(),a.defaultPrevented}();if(!c){var d=Event.prototype.preventDefault;Event.prototype.preventDefault=function(){this.cancelable&&(d.call(this),Object.defineProperty(this,"defaultPrevented",{get:function(){return!0}}))}}var e=/Trident/.test(navigator.userAgent);if((!window.CustomEvent||e&&"function"!=typeof window.CustomEvent)&&(window.CustomEvent=function(a,b){b=b||{};var c=document.createEvent("CustomEvent");return c.initCustomEvent(a,Boolean(b.bubbles),Boolean(b.cancelable),b.detail),c},window.CustomEvent.prototype=window.Event.prototype),!window.Event||e&&"function"!=typeof window.Event){var f=window.Event;window.Event=function(a,b){b=b||{};var c=document.createEvent("Event");return c.initEvent(a,Boolean(b.bubbles),Boolean(b.cancelable)),c},window.Event.prototype=f.prototype}}(window.WebComponents),window.HTMLImports=window.HTMLImports||{flags:{}},function(a){function b(a,b){b=b||o,d(function(){f(a,b)},b)}function c(a){return"complete"===a.readyState||a.readyState===r}function d(a,b){if(c(b))a&&a();else{var e=function(){("complete"===b.readyState||b.readyState===r)&&(b.removeEventListener(s,e),d(a,b))};b.addEventListener(s,e)}}function e(a){a.target.__loaded=!0}function f(a,b){function c(){i==j&&a&&a({allImports:h,loadedImports:k,errorImports:l})}function d(a){e(a),k.push(this),i++,c()}function f(a){l.push(this),i++,c()}var h=b.querySelectorAll("link[rel=import]"),i=0,j=h.length,k=[],l=[];if(j)for(var m,n=0;j>n&&(m=h[n]);n++)g(m)?(i++,c()):(m.addEventListener("load",d),m.addEventListener("error",f));else c()}function g(a){return l?a.__loaded||a.import&&"loading"!==a.import.readyState:a.__importParsed}function h(a){for(var b,c=0,d=a.length;d>c&&(b=a[c]);c++)i(b)&&j(b)}function i(a){return"link"===a.localName&&"import"===a.rel}function j(a){var b=a.import;b?e({target:a}):(a.addEventListener("load",e),a.addEventListener("error",e))}var k="import",l=Boolean(k in document.createElement("link")),m=Boolean(window.ShadowDOMPolyfill),n=function(a){return m?window.ShadowDOMPolyfill.wrapIfNeeded(a):a},o=n(document),p={get:function(){var a=window.HTMLImports.currentScript||document.currentScript||("complete"!==document.readyState?document.scripts[document.scripts.length-1]:null);return n(a)},configurable:!0};Object.defineProperty(document,"_currentScript",p),Object.defineProperty(o,"_currentScript",p);var q=/Trident/.test(navigator.userAgent),r=q?"complete":"interactive",s="readystatechange";l&&(new MutationObserver(function(a){for(var b,c=0,d=a.length;d>c&&(b=a[c]);c++)b.addedNodes&&h(b.addedNodes)}).observe(document.head,{childList:!0}),function(){if("loading"===document.readyState)for(var a,b=document.querySelectorAll("link[rel=import]"),c=0,d=b.length;d>c&&(a=b[c]);c++)j(a)}()),b(function(a){window.HTMLImports.ready=!0,window.HTMLImports.readyTime=(new Date).getTime();var b=o.createEvent("CustomEvent");b.initCustomEvent("HTMLImportsLoaded",!0,!0,a),o.dispatchEvent(b)}),a.IMPORT_LINK_TYPE=k,a.useNative=l,a.rootDocument=o,a.whenReady=b,a.isIE=q}(window.HTMLImports),function(a){var b=[],c=function(a){b.push(a)},d=function(){b.forEach(function(b){b(a)})};a.addModule=c,a.initializeModules=d}(window.HTMLImports),window.HTMLImports.addModule(function(a){var b=/(url\()([^)]*)(\))/g,c=/(@import[\s]+(?!url\())([^;]*)(;)/g,d={resolveUrlsInStyle:function(a,b){var c=a.ownerDocument,d=c.createElement("a");return a.textContent=this.resolveUrlsInCssText(a.textContent,b,d),a},resolveUrlsInCssText:function(a,d,e){var f=this.replaceUrls(a,e,d,b);return f=this.replaceUrls(f,e,d,c)},replaceUrls:function(a,b,c,d){return a.replace(d,function(a,d,e,f){var g=e.replace(/["']/g,"");return c&&(g=new URL(g,c).href),b.href=g,g=b.href,d+"'"+g+"'"+f})}};a.path=d}),window.HTMLImports.addModule(function(a){var b={async:!0,ok:function(a){return a.status>=200&&a.status<300||304===a.status||0===a.status},load:function(c,d,e){var f=new XMLHttpRequest;return(a.flags.debug||a.flags.bust)&&(c+="?"+Math.random()),f.open("GET",c,b.async),f.addEventListener("readystatechange",function(a){if(4===f.readyState){var c=null;try{var g=f.getResponseHeader("Location");g&&(c="/"===g.substr(0,1)?location.origin+g:g)}catch(a){console.error(a.message)}d.call(e,!b.ok(f)&&f,f.response||f.responseText,c)}}),f.send(),f},loadDocument:function(a,b,c){this.load(a,b,c).responseType="document"}};a.xhr=b}),window.HTMLImports.addModule(function(a){var b=a.xhr,c=a.flags,d=function(a,b){this.cache={},this.onload=a,this.oncomplete=b,this.inflight=0,this.pending={}};d.prototype={addNodes:function(a){this.inflight+=a.length;for(var b,c=0,d=a.length;d>c&&(b=a[c]);c++)this.require(b);this.checkDone()},addNode:function(a){this.inflight++,this.require(a),this.checkDone()},require:function(a){var b=a.src||a.href;a.__nodeUrl=b,this.dedupe(b,a)||this.fetch(b,a)},dedupe:function(a,b){return this.pending[a]?(this.pending[a].push(b),!0):this.cache[a]?(this.onload(a,b,this.cache[a]),this.tail(),!0):(this.pending[a]=[b],!1)},fetch:function(a,d){if(c.load&&console.log("fetch",a,d),a)if(a.match(/^data:/)){var e=a.split(","),f=e[0],g=e[1];g=f.indexOf(";base64")>-1?atob(g):decodeURIComponent(g),setTimeout(function(){this.receive(a,d,null,g)}.bind(this),0)}else{var h=function(b,c,e){this.receive(a,d,b,c,e)}.bind(this);b.load(a,h)}else setTimeout(function(){this.receive(a,d,{error:"href must be specified"},null)}.bind(this),0)},receive:function(a,b,c,d,e){this.cache[a]=d;for(var f,g=this.pending[a],h=0,i=g.length;i>h&&(f=g[h]);h++)this.onload(a,f,d,c,e),this.tail();this.pending[a]=null},tail:function(){--this.inflight,this.checkDone()},checkDone:function(){this.inflight||this.oncomplete()}},a.Loader=d}),window.HTMLImports.addModule(function(a){var b=function(a){this.addCallback=a,this.mo=new MutationObserver(this.handler.bind(this))};b.prototype={handler:function(a){for(var b,c=0,d=a.length;d>c&&(b=a[c]);c++)"childList"===b.type&&b.addedNodes.length&&this.addedNodes(b.addedNodes)},addedNodes:function(a){this.addCallback&&this.addCallback(a);for(var b,c=0,d=a.length;d>c&&(b=a[c]);c++)b.children&&b.children.length&&this.addedNodes(b.children)},observe:function(a){this.mo.observe(a,{childList:!0,subtree:!0})}},a.Observer=b}),window.HTMLImports.addModule(function(a){function b(a){return"link"===a.localName&&a.rel===k}function c(a){var b=d(a);return"data:text/javascript;charset=utf-8,"+encodeURIComponent(b)}function d(a){return a.textContent+e(a)}function e(a){var b=a.ownerDocument;b.__importedScripts=b.__importedScripts||0;var c=a.ownerDocument.baseURI,d=b.__importedScripts?"-"+b.__importedScripts:"";return b.__importedScripts++,"\n//# sourceURL="+c+d+".js\n"}function f(a){var b=a.ownerDocument.createElement("style");return b.textContent=a.textContent,g.resolveUrlsInStyle(b),b}var g=a.path,h=a.rootDocument,i=a.flags,j=a.isIE,k=a.IMPORT_LINK_TYPE,l="link[rel="+k+"]",m={documentSelectors:l,importsSelectors:[l,"link[rel=stylesheet]:not([type])","style:not([type])","script:not([type])",'script[type="application/javascript"]','script[type="text/javascript"]'].join(","),map:{link:"parseLink",script:"parseScript",style:"parseStyle"},dynamicElements:[],parseNext:function(){var a=this.nextToParse();a&&this.parse(a)},parse:function(a){if(this.isParsed(a))return void(i.parse&&console.log("[%s] is already parsed",a.localName));var b=this[this.map[a.localName]];b&&(this.markParsing(a),b.call(this,a))},parseDynamic:function(a,b){this.dynamicElements.push(a),b||this.parseNext()},markParsing:function(a){i.parse&&console.log("parsing",a),this.parsingElement=a},markParsingComplete:function(a){a.__importParsed=!0,this.markDynamicParsingComplete(a),a.__importElement&&(a.__importElement.__importParsed=!0,this.markDynamicParsingComplete(a.__importElement)),this.parsingElement=null,i.parse&&console.log("completed",a)},markDynamicParsingComplete:function(a){var b=this.dynamicElements.indexOf(a);b>=0&&this.dynamicElements.splice(b,1)},parseImport:function(a){if(a.import=a.__doc,window.HTMLImports.__importsParsingHook&&window.HTMLImports.__importsParsingHook(a),a.import&&(a.import.__importParsed=!0),this.markParsingComplete(a),a.__resource&&!a.__error?a.dispatchEvent(new CustomEvent("load",{bubbles:!1})):a.dispatchEvent(new CustomEvent("error",{bubbles:!1})),a.__pending)for(var b;a.__pending.length;)b=a.__pending.shift(),b&&b({target:a});this.parseNext()},parseLink:function(a){b(a)?this.parseImport(a):(a.href=a.href,this.parseGeneric(a))},parseStyle:function(a){var b=a;a=f(a),b.__appliedElement=a,a.__importElement=b,this.parseGeneric(a)},parseGeneric:function(a){this.trackElement(a),this.addElementToDocument(a)},rootImportForElement:function(a){for(var b=a;b.ownerDocument.__importLink;)b=b.ownerDocument.__importLink;return b},addElementToDocument:function(a){var b=this.rootImportForElement(a.__importElement||a);b.parentNode.insertBefore(a,b)},trackElement:function(a,b){var c=this,d=function(e){a.removeEventListener("load",d),a.removeEventListener("error",d),b&&b(e),c.markParsingComplete(a),c.parseNext()};if(a.addEventListener("load",d),a.addEventListener("error",d),j&&"style"===a.localName){var e=!1;if(-1==a.textContent.indexOf("@import"))e=!0;else if(a.sheet){e=!0;for(var f,g=a.sheet.cssRules,h=g?g.length:0,i=0;h>i&&(f=g[i]);i++)f.type===CSSRule.IMPORT_RULE&&(e=e&&Boolean(f.styleSheet))}e&&setTimeout(function(){a.dispatchEvent(new CustomEvent("load",{bubbles:!1}))})}},parseScript:function(b){var d=document.createElement("script");d.__importElement=b,d.src=b.src?b.src:c(b),a.currentScript=b,this.trackElement(d,function(b){d.parentNode&&d.parentNode.removeChild(d),a.currentScript=null}),this.addElementToDocument(d)},nextToParse:function(){return this._mayParse=[],!this.parsingElement&&(this.nextToParseInDoc(h)||this.nextToParseDynamic())},nextToParseInDoc:function(a,c){if(a&&this._mayParse.indexOf(a)<0){this._mayParse.push(a);for(var d,e=a.querySelectorAll(this.parseSelectorsForNode(a)),f=0,g=e.length;g>f&&(d=e[f]);f++)if(!this.isParsed(d))return this.hasResource(d)?b(d)?this.nextToParseInDoc(d.__doc,d):d:void 0}return c},nextToParseDynamic:function(){return this.dynamicElements[0]},parseSelectorsForNode:function(a){var b=a.ownerDocument||a;return b===h?this.documentSelectors:this.importsSelectors},isParsed:function(a){return a.__importParsed},needsDynamicParsing:function(a){return this.dynamicElements.indexOf(a)>=0},hasResource:function(a){return!b(a)||void 0!==a.__doc}};a.parser=m,a.IMPORT_SELECTOR=l}),window.HTMLImports.addModule(function(a){function b(a){return c(a,g)}function c(a,b){return"link"===a.localName&&a.getAttribute("rel")===b}function d(a){return!!Object.getOwnPropertyDescriptor(a,"baseURI")}function e(a,b){var c=document.implementation.createHTMLDocument(g);c._URL=b;var e=c.createElement("base");e.setAttribute("href",b),c.baseURI||d(c)||Object.defineProperty(c,"baseURI",{value:b});var f=c.createElement("meta");return f.setAttribute("charset","utf-8"),c.head.appendChild(f),c.head.appendChild(e),c.body.innerHTML=a,window.HTMLTemplateElement&&HTMLTemplateElement.bootstrap&&HTMLTemplateElement.bootstrap(c),c}var f=a.flags,g=a.IMPORT_LINK_TYPE,h=a.IMPORT_SELECTOR,i=a.rootDocument,j=a.Loader,k=a.Observer,l=a.parser,m={documents:{},documentPreloadSelectors:h,importsPreloadSelectors:[h].join(","),loadNode:function(a){n.addNode(a)},loadSubtree:function(a){var b=this.marshalNodes(a);n.addNodes(b)},marshalNodes:function(a){return a.querySelectorAll(this.loadSelectorsForNode(a))},loadSelectorsForNode:function(a){var b=a.ownerDocument||a;return b===i?this.documentPreloadSelectors:this.importsPreloadSelectors},loaded:function(a,c,d,g,h){if(f.load&&console.log("loaded",a,c),c.__resource=d,c.__error=g,b(c)){var i=this.documents[a];void 0===i&&(i=g?null:e(d,h||a),i&&(i.__importLink=c,this.bootDocument(i)),this.documents[a]=i),c.__doc=i}l.parseNext()},bootDocument:function(a){this.loadSubtree(a),this.observer.observe(a),l.parseNext()},loadedAll:function(){l.parseNext()}},n=new j(m.loaded.bind(m),m.loadedAll.bind(m));if(m.observer=new k,!document.baseURI){var o={get:function(){var a=document.querySelector("base");return a?a.href:window.location.href},configurable:!0};Object.defineProperty(document,"baseURI",o),Object.defineProperty(i,"baseURI",o)}a.importer=m,a.importLoader=n}),window.HTMLImports.addModule(function(a){var b=a.parser,c=a.importer,d={added:function(a){for(var d,e,f,g,h=0,i=a.length;i>h&&(g=a[h]);h++)d||(d=g.ownerDocument,e=b.isParsed(d)),f=this.shouldLoadNode(g),f&&c.loadNode(g),this.shouldParseNode(g)&&e&&b.parseDynamic(g,f)},shouldLoadNode:function(a){return 1===a.nodeType&&e.call(a,c.loadSelectorsForNode(a))},shouldParseNode:function(a){return 1===a.nodeType&&e.call(a,b.parseSelectorsForNode(a))}};c.observer.addCallback=d.added.bind(d);var e=HTMLElement.prototype.matches||HTMLElement.prototype.matchesSelector||HTMLElement.prototype.webkitMatchesSelector||HTMLElement.prototype.mozMatchesSelector||HTMLElement.prototype.msMatchesSelector}),function(a){function b(){window.HTMLImports.importer.bootDocument(d)}var c=a.initializeModules;if(a.isIE,!a.useNative){c();var d=a.rootDocument;"complete"===document.readyState||"interactive"===document.readyState&&!window.attachEvent?b():document.addEventListener("DOMContentLoaded",b)}}(window.HTMLImports),window.CustomElements=window.CustomElements||{flags:{}},function(a){var b=a.flags,c=[],d=function(a){c.push(a)},e=function(){c.forEach(function(b){b(a)})};a.addModule=d,a.initializeModules=e,a.hasNative=Boolean(document.registerElement),a.isIE=/Trident/.test(navigator.userAgent),a.useNative=!b.register&&a.hasNative&&!window.ShadowDOMPolyfill&&(!window.HTMLImports||window.HTMLImports.useNative)}(window.CustomElements),window.CustomElements.addModule(function(a){function b(a,b){c(a,function(a){return!!b(a)||void d(a,b)}),d(a,b)}function c(a,b,d){var e=a.firstElementChild;if(!e)for(e=a.firstChild;e&&e.nodeType!==Node.ELEMENT_NODE;)e=e.nextSibling;for(;e;)b(e,d)!==!0&&c(e,b,d),e=e.nextElementSibling;return null}function d(a,c){for(var d=a.shadowRoot;d;)b(d,c),d=d.olderShadowRoot}function e(a,b){f(a,b,[])}function f(a,b,c){if(a=window.wrap(a),!(c.indexOf(a)>=0)){c.push(a);for(var d,e=a.querySelectorAll("link[rel="+g+"]"),h=0,i=e.length;i>h&&(d=e[h]);h++)d.import&&f(d.import,b,c);b(a)}}var g=window.HTMLImports?window.HTMLImports.IMPORT_LINK_TYPE:"none";a.forDocumentTree=e,a.forSubtree=b}),window.CustomElements.addModule(function(a){function b(a,b){return c(a,b)||d(a,b)}function c(b,c){return!!a.upgrade(b,c)||void(c&&g(b))}function d(a,b){t(a,function(a){return!!c(a,b)||void 0})}function e(a){x.push(a),w||(w=!0,setTimeout(f))}function f(){w=!1;for(var a,b=x,c=0,d=b.length;d>c&&(a=b[c]);c++)a();x=[]}function g(a){v?e(function(){h(a)}):h(a)}function h(a){a.__upgraded__&&!a.__attached&&(a.__attached=!0,a.attachedCallback&&a.attachedCallback())}function i(a){j(a),t(a,function(a){j(a)})}function j(a){v?e(function(){k(a)}):k(a)}function k(a){a.__upgraded__&&a.__attached&&(a.__attached=!1,a.detachedCallback&&a.detachedCallback())}function l(a){for(var b=a,c=window.wrap(document);b;){if(b==c)return!0;b=b.parentNode||b.nodeType===Node.DOCUMENT_FRAGMENT_NODE&&b.host}}function m(a){if(a.shadowRoot&&!a.shadowRoot.__watched){s.dom&&console.log("watching shadow-root for: ",a.localName);for(var b=a.shadowRoot;b;)p(b),b=b.olderShadowRoot}}function n(a,c){if(s.dom){var d=c[0];if(d&&"childList"===d.type&&d.addedNodes&&d.addedNodes){for(var e=d.addedNodes[0];e&&e!==document&&!e.host;)e=e.parentNode;var f=e&&(e.URL||e._URL||e.host&&e.host.localName)||"";f=f.split("/?").shift().split("/").pop()}console.group("mutations (%d) [%s]",c.length,f||"")}var g=l(a);c.forEach(function(a){"childList"===a.type&&(y(a.addedNodes,function(a){a.localName&&b(a,g)}),y(a.removedNodes,function(a){a.localName&&i(a)}))}),s.dom&&console.groupEnd()}function o(a){for(a=window.wrap(a),a||(a=window.wrap(document));a.parentNode;)a=a.parentNode;var b=a.__observer;b&&(n(a,b.takeRecords()),f())}function p(a){if(!a.__observer){var b=new MutationObserver(n.bind(this,a));b.observe(a,{childList:!0,subtree:!0}),a.__observer=b}}function q(a){a=window.wrap(a),s.dom&&console.group("upgradeDocument: ",a.baseURI.split("/").pop());var c=a===window.wrap(document);b(a,c),p(a),s.dom&&console.groupEnd();


Do these TF static files come directly from the TF repo? If so, we need copyright info on them. I believe this is the appropriate incantation:

Copyright (c) 2017, NVIDIA CORPORATION. All rights reserved. Copyright 2017 The TensorFlow Authors. All rights reserved.

Also, if there's a non-minimized version of this file, that would be preferable (even though the total LOC for this PR would skyrocket even higher!).

@ethantang95 can you add those copyright lines? Just leave the minification. I did include the copyright files in digits/static/tb/... afaik.

lukeyeager · 2017-07-11T18:53:58Z

scripts/travis/install-torch.sh

@@ -42,4 +42,3 @@ LOG_FILE="$LOCAL_DIR/torch-install.log"
 # mark cache
 WEEK=$(date +%Y-%W)
 echo "$WEEK" >"${INSTALL_DIR}/cache-version.txt"
-


Unnecessary

lukeyeager · 2017-07-11T18:54:03Z

scripts/travis/ppa-upload.sh

@@ -28,3 +28,5 @@ cd "$ROOT_DIR/packaging/deb/dist/"
 cd ./*xenial/
 debsign -k 97A4B458 ./*source.changes
 dput -U "ppa:nvidia-digits/${PPA_NAME}/ubuntu/xenial" ./*source.changes
+
+echo "ppa-upload.sh finished"


Unnecessary

lukeyeager · 2017-07-11T18:54:07Z

scripts/travis/pypi-upload.sh

@@ -15,3 +15,5 @@ username = luke.yeager
 password = ${PYPI_PASSWORD}
 EOF
 twine upload -r pypi dist/*
+
+echo "pypi-upload.sh finished"


Unnecessary

lukeyeager · 2017-07-11T18:54:16Z

setup.cfg


 [pep8]
 max-line-length = 120
 exclude = venv
+


Unnecessary

lukeyeager · 2017-07-11T18:54:40Z

setup.cfg

@@ -1,7 +1,8 @@
 [flake8]
 max-line-length = 120
-exclude = venv
+exclude = venv,standard-networks


Can we check the standard networks, too?

lukeyeager · 2017-07-13T16:14:22Z

@ethantang95 I merged #1723. Try rebasing this branch on origin/master and see if TravisCI works now.

lukeyeager · 2017-07-13T20:46:18Z

result.txt

@@ -0,0 +1,470 @@
+Level 1:tensorflow:Registering FakeQuantWithMinMaxArgs (<function _FakeQuantWithMinMaxArgsGradient at 0x7f177ce7e140>) in gradient.


Don't add this file.

Whoops, forgot to delete that after running the tests

lukeyeager · 2017-07-13T23:39:27Z

TODOs:

Remove some unnecessary whitespace changes
Replace unslim standard networks with slim ones (and make sure they still work)
Get rid of the ridiculously verbose tensorflow logging messages in the tests (maybe start here?)

Then I'll approve, then you can squash all your changes (but not Tim's or Greg's), then we can merge.

ethantang95 · 2017-07-14T00:50:26Z

Suppressing the level 1 logs output by tensorflow is impossible as per tensorflow/tensorflow#8340

Also we want to have the debug logs which I think @TimZaman uses it to parse and get the learning status

gheinrich · 2017-07-14T13:58:09Z

@ethantang95 one thing you might want to check: if you can find some info on .data-00000-of-00001 and whether TF will create more shards per snapshot if the model is huge. In which case the logic in the code might have to be revisited.

ethantang95 · 2017-07-14T17:22:53Z

So I looked around and asked a few others that uses tensorflow, none of them have seen it actually shard yet even with 16gb files... so I guess it's safe to say that it would be really hard to get sharding to actually happen and I guess we can deal with it when someone actually hit that case.

TimZaman · 2017-07-14T17:23:47Z

I don't think it shards.

…

On Fri, Jul 14, 2017 at 7:23 PM, Ethan Tang ***@***.***> wrote: So I looked around and asked a few others that uses tensorflow, none of them have seen it actually shard yet even with 16gb files... so I guess it's safe to say that it would be really hard to get sharding to actually happen and I guess we can deal with it when someone actually hit that case. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#1714 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AHXSRHk_4IOt4TYrzCzrZWUYrbjF49uvks5sN6P0gaJpZM4OP_MT> .

TimZaman · 2017-07-17T16:24:27Z

Depening on if the slim networks give better performance, we might want to reintroduce those instead of the vanilla alexnet, f.e.

…

On Mon, 17 Jul 2017 at 18:22, Luke Yeager ***@***.***> wrote: ***@***.**** approved this pull request. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#1714 (review)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AHXSRHeRsmk14KrKqo7WJ4RgE92gg1MWks5sO4kngaJpZM4OP_MT> .

TimZaman · 2017-07-17T16:25:11Z

Want to merge? Note this isnt to master yet.

lukeyeager · 2017-07-17T16:55:20Z

Looks like TravisCI is failing for reasons not related to this PR: pypa/setuptools#1086

Added some initial tf tools Implemented UI Fixes for tensorflow 0.10 Removed tf-slim as its not part of the 0.10 master Added the lmdb reader with a tf.cond that needs replacement Implemented train and val seperation with a templating Fixed issue with dequeueing both runners by pulling both graphs Implemented training and validation rythm Added support for both png and jpg and added 16 bit support Implemented mean subtraction - but needs rework to load as constant Added an optimized implementation of mean subtraction Further optimized the mean loading by using a shared constant Wrapped the data loader in a factory to easily support more data types Implemented cropping Implemented floating point support. Implemented seperate LMDB database. Implemented regression support. Added some brief nosetests. Need to invoke accuracy only on classification though. Implemented variable restoration. Needs thorough testing Implemented inferencing, not entirely polished Moved some code into functions, started on modularization a bit Implemented digits custom helper functions Implemented custom printing ops Implemented autoencoder total rewrite of summaries Implemented output to console from scalar summaries Fixes for summary outputs: only simple scalar values are parsed to console Implemented binary segmentation and necessary fixes Some updates on binary seg Implemented all possible optimizers and started work on learning rate shaper Started work on the lr policies Fixes for learning_rates, implemented optimizers, tested variable summary output to UI Implemented and tested all learning rates and optimizers Introduces new model definition and improvements in loss handling and graph layout Major refactoring of main code. Implemented new model description. Implemented and tested inferencing. Implemented and tested weight/snapshot loading. All-round minor updates and fixes Fixes in summary cumulator and implemented an RNN model Fixes for mean subtraction in tf and tf-ui, implemented data order selection in image-view extension Implemented support for mean file of format: png, jpg, binaryproto - the latter being the fault that DIGITS will provide. Added support for runtime statistics and some allround fixes Added static tensorboard style network visualization for tensorflow. Added output of traces (no vis yet). Added a loader while waiting for network vis. Minor syntax cleanups. Implemented alexnet standard network Pulled in updates for travis build and added tensorflow install Added two more files for Mr Travis Implemented tensorflow configuration Added tf config to doc Fixes for ubuntu deployment of tf. Moved tf tools Fixes for tf ubuntu Fixes for tf ubuntu Some fixes and updates for TF in Travis Fix in network viz test Implemented default sinlge-gpu support and some nosetests Fixes for inference Added siamese network, bugfixes, minor features, some utility tf functions Added siamese network and example png Better error-ui format for network viz Added an alternative simpler siamese network that doesnt need a seperate db, minor error update Preliminary version of hdf5 implemented Implemented fine-tuning by renaming variables Implemented visualisation of variables and the activations of the Ops they belong to. Fix in inf vis naming Fixes in visualualisation shapes and naming Implemented softmax upon classification Implemented all nosetests for tf classification, and many allround bugfixes Implemented generic nosetests - some need work Fix for travis to find python exe Implemented a better file format deducer, and implemented a bare minimal TFRecord-reader Added top_n accuracy shortcut Implemented on-line data augmentation for TF, 5 types. Some minor bugfixes. Need to do something with image whitening though during validation and inf.. Added tensorflow data augmentation test Minor fixes and improvements from linter Implemented minimal and bare multigpu and fixes to get it running for greg Preliminary version of tfrecord writer for classification Some changes to optimize dataloading for tfr More fixes for tfrecrods Fix generic data loading Minor breaking changes but updates in namescoping Implemented new model structure. Improvements to multi-gpu handling. Updates to namespaces. Implemented accounting for regularization. Many allround updates Implemented proper visualisation for gpu devices Minor updates and converted alexnet and vgg16 to new format Fix in tfrecord shape WIP on timeline traces Finalized support for tensorflow timeline traces Fixed alexnet for tf Fix merge errors Minify tf-graph-basic.build.js

bAbI data plug-in Add utils Add inference form to bAbI dataset Allow inference without answer Allow unknown words in BaBI data plug-in Fix bAbI plugin Lint errors Tensorflow integration updates Use TFRecords for TF inference TF: Don't rescale inputs Fix some TF classification tests Remove unnecessary print Fix TF imports when uninstalled Fix mean image scale Fix generic model tests Fix Torch single image inference Fix inference TMP TF Lint Revert changes in digits-lint script Lint: ignore tensorflow standard examples More Lint fixes Add gradient hook Add memn2n model Update memn2n with gradient hooks GAN example Make batch size variable Training/inference paths Small update to TF 0.12 Snapshot names, float inference, restore all vars Do not restore global_step or optimizer variables Add TB link Update GAN network Dynamically select inference form TF inference: convert images to float Update GAN z-gen network Small Update model view layout Add GAN plug-ins Update GAN plug-in to create CelebA dataset Add ability to show input in ImageOutput extension Add all data to raw data view extension Add model for CelebA dataset Update GAN data plug-in Update all losses in one session Remove conversion to .png in GAN data plug-in TF Slim Lenet example Divide input by 255 Update GAN data plug-in Fix TF model snapshot Reduce scheduler delays to speed up inference Update GAN plugins Fix TF tests Add API to LmdbReader (used by gan_features.py) Save animated gif Add GAN walk-through Update GAN walkthrough with embeddings video Fix GAN view for list encoding Add animation task to GAN plugins Add view task to see image attributes Add comments to GAN models Update README Fix GAN features script GAN app Fix DIGITS inference Adjust GAN window size automatically Add attributes to GAN app Move gandisplay.py Remove wxpython 3.0 selection Fix call to model Adding disclaimer

some small tidying changes WRT to pull request lint fixes re-enabled disabled tests changes WRT pull request comments Travis bug fixes Testing a patch fixed travis bugs lint

changes wrt to PR comments Updated version number updated version number to 6.0 changed version number to proper format updated manifest to fix bug 200328271 fixed typo in build tensorflow doc for bug 200328296 changes WRT PR comments changes WRT PR comments accidentally added test result file removed dependency to the tensorflow environment variable changed from 'python' to using sys.executable changing version back to 5.1-dev for travis whitespace fixes white space changes replaced the tensorflow networks with slim models Removed tests referencing lenet slim

lukeyeager · 2017-07-17T21:10:32Z

Thanks @TimZaman for the (massive) initial work, @gheinrich for building an interesting workflow on top of Tim's work, and @ethantang95 for pulling it all together and bringing us to the finish line!

TimZaman · 2017-07-17T21:31:46Z

And you, for DIGITS!

…

On Mon, 17 Jul 2017 at 23:10, Luke Yeager ***@***.***> wrote: Merged #1714 <#1714>. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#1714 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AHXSRH1CAKl4s53ZiYJLomr_qfxns21Vks5sO83ggaJpZM4OP_MT> .

ethantang95 requested review from lukeyeager, gheinrich and TimZaman July 6, 2017 18:05

TimZaman mentioned this pull request Jul 9, 2017

Updated to work with TF 1.0 TimZaman/DIGITS#2

Closed

TimZaman self-assigned this Jul 9, 2017

TimZaman reviewed Jul 9, 2017

View reviewed changes

gheinrich requested changes Jul 10, 2017

View reviewed changes

ethantang95 force-pushed the pr-request-2 branch 6 times, most recently from fbcb7c6 to 45a51d9 Compare July 11, 2017 00:11

lukeyeager suggested changes Jul 11, 2017

View reviewed changes

lukeyeager mentioned this pull request Jul 12, 2017

TravisCI: build+install protobuf3 #1723

Merged

lukeyeager added the tensorflow label Jul 12, 2017

ethantang95 force-pushed the pr-request-2 branch 2 times, most recently from c905acc to ff77a8b Compare July 12, 2017 23:37

NVIDIA deleted a comment from TimZaman Jul 13, 2017

lukeyeager reviewed Jul 13, 2017

View reviewed changes

ethantang95 changed the base branch from tensorflow to master July 13, 2017 22:43

gheinrich approved these changes Jul 13, 2017

View reviewed changes

ethantang95 force-pushed the pr-request-2 branch from 8de2a97 to 2a17c4d Compare July 14, 2017 22:39

lukeyeager approved these changes Jul 17, 2017

View reviewed changes

ethantang95 force-pushed the pr-request-2 branch from 42debf0 to 42c9858 Compare July 17, 2017 17:27

TimZaman and others added 4 commits July 17, 2017 12:54

Preparing Tensorflow branch for upstream merge

7a2c3a2

some small tidying changes WRT to pull request lint fixes re-enabled disabled tests changes WRT pull request comments Travis bug fixes Testing a patch fixed travis bugs lint

ethantang95 force-pushed the pr-request-2 branch from 42c9858 to 7092833 Compare July 17, 2017 19:55

added comments for the preferred training options for alex and googlenet

0be9e7b

lukeyeager merged commit 016d7c0 into NVIDIA:master Jul 17, 2017

		super(TestTensorflowCreatedWithGradientDataExtensionNoValSet, cls).setUpClass(val_image_count=0)


		# class TestTensorflowCreatedWithImageProcessingExtensionMeanImage(BaseTestCreatedWithImageProcessingExtension,

		@@ -4,6 +4,8 @@

		DIGITS (the Deep Learning GPU Training System) is a webapp for training deep learning models.

		The currently supported frameworks are: Caffe 1, Torch, and Tensorflow

		@@ -0,0 +1,470 @@
		Level 1:tensorflow:Registering FakeQuantWithMinMaxArgs (<function _FakeQuantWithMinMaxArgsGradient at 0x7f177ce7e140>) in gradient.

Tensorflow support for DIGITS #1714

Tensorflow support for DIGITS #1714

Conversation

ethantang95 commented Jul 6, 2017

TimZaman left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

TimZaman commented Jul 9, 2017 • edited Loading

TimZaman commented Jul 9, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

TimZaman Jul 11, 2017 • edited Loading

Choose a reason for hiding this comment

TimZaman commented Jul 9, 2017 • edited Loading

gheinrich commented Jul 10, 2017

TimZaman commented Jul 10, 2017 via email

gheinrich left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

gheinrich commented Jul 10, 2017

gheinrich commented Jul 10, 2017

lukeyeager commented Jul 11, 2017

lukeyeager left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ethantang95 Jul 11, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ethantang95 Jul 12, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

gheinrich Jul 13, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

lukeyeager commented Jul 11, 2017

ethantang95 commented Jul 11, 2017

lukeyeager left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

TimZaman commented Jul 9, 2017 •

edited

Loading

TimZaman Jul 11, 2017 •

edited

Loading

TimZaman commented Jul 9, 2017 •

edited

Loading

ethantang95 Jul 11, 2017 •

edited

Loading

ethantang95 Jul 12, 2017 •

edited

Loading

gheinrich Jul 13, 2017 •

edited

Loading