new demo site with integrated capture and graph/download, plus Cypress tests #175

TonyIversonLin · 2020-09-05T00:53:57Z

Make sure these boxes are checked before your pull request is ready to be reviewed and merged. Thanks!

tests pass -- see README.md for how to run them
code is in uniquely-named feature branch, and has been rebased on top of latest master (especially if you've been asked to make additional changes)
pull request are descriptively named
if possible, multiple commits squashed if they're smaller changes
reviewed/confirmed/tested by another contributor or maintainer

Please be sure you've reviewed our contribution guidelines at https://publiclab.org/wiki/contributing-to-public-lab-software

Please alert developers on plots-dev@googlegroups.com when your request is ready or if you need assistance.

Thanks!

Yechao change

…g step

welcome · 2020-09-05T00:53:59Z

Thanks for opening this pull request! Dangerbot will test out your code and reply in a bit with some pointers and requests.
There may be some errors, but don't worry! We're here to help! 👍🎉😄

gitpod-io · 2020-09-05T00:54:01Z

jywarren · 2020-09-17T16:37:10Z

examples/new-capture/index_copy.html

@@ -0,0 +1,363 @@
+<!DOCTYPE html>


Just wondering how this is distinct from index.html? Thank you!

jywarren

This looks fantastic! A couple small notes and requests:

I noticed a .DS_Store file was added, probably inadvertently - can you just remove that? Thanks!
The copy of the new-capture JS files seems like it could diverge development of those files, which may be changing quickly. Was it intentional to make 2 copies of each, and are they exact copies?
The new-capture index.html is still pointed at dist/capture.dist.js, and that file is compiled from the examples/capture/*.js files, not the new new-capture copies, so i believe the new copies are not being used in this demo, is that correct? Below is the Gruntfile code which compiles the files:

spectral-workbench.js/Gruntfile.js

Lines 49 to 57 in 5b176c5

    
           capture: { 
        
               src: [ 
        
                   'examples/capture/detect_sample_row.js', 
        
                   'examples/capture/toggleRotation.js', 
        
                   'examples/capture/getRow.js', 
        
                   'examples/capture/getUserMedia.js' 
        
               ], 
        
               dest: 'dist/capture.dist.js' 
        
           }

Thanks, just need a little clarity on these but overall this is looking really fantastic. I love your final report as well, would you be able to share that in this PR somehow for other contributors to read?

Great work, we're almost there!! Many thanks!

jywarren · 2020-09-17T16:40:59Z

examples/new-capture/index.html

+
+
+  <script src="capture.js" type="text/javascript"></script>
+  <script src="../../dist/capture.dist.js" type="text/javascript"></script>


This is pointed at the compiled file, see review note about Grunt compilation! Thanks!

jywarren · 2020-09-17T16:46:45Z

I'm also seeing an odd error in our Travis build:

27.02s$ npm ci 
> cypress@5.1.0 postinstall /home/travis/build/publiclab/spectral-workbench.js/node_modules/cypress
> node index.js --exec install
> cypress@5.1.0 postinstall /home/travis/build/publiclab/spectral-workbench.js/node_modules/cypress
> node index.js --exec install
/home/travis/build/publiclab/spectral-workbench.js/node_modules/cypress/node_modules/fs-extra/lib/mkdirs/make-dir.js:85
      } catch {
              ^
SyntaxError: Unexpected token {
    at createScript (vm.js:80:10)
    at Object.runInThisContext (vm.js:139:10)
    at Module._compile (module.js:617:28)
    at Object.Module._extensions..js (module.js:664:10)

https://travis-ci.com/github/publiclab/spectral-workbench.js/builds/182860297#L399-L407

This seems related to the npm ci command and/or the postinstall stage of cypress. Does this happen to you on local?

jywarren · 2020-09-17T16:50:51Z

And, just confirming that the new capture demo works! Presumably using the compiled (and previous) capture code:

A couple icons aren't showing, but it's a minor issue!

jywarren · 2020-09-17T16:53:54Z

Ah, i am getting an error and no graph on the final "save graph" step:

It looks like d3 is not getting loaded?

Uncaught ReferenceError: d3 is not defined
    at Class.init (spectral-workbench.js:5178)
    at new Class (spectral-workbench.js:62)

[2020-09-17T16:48:25.081Z] "GET /bower_components/d3/d3.js" Error (404): "Not found"

Perhaps it has to be included via NPM instead of bower -- the correct route should be /node_modules/d3/d3.js, no?

jywarren · 2020-09-17T16:55:46Z

bower.json

+    "moment": "~2.10.6",
+    "nvd3": "~1.7.1",
+    "d3": "~3.3.13",
+    "jquery.steps": "^1.1.0"


Here I think we should move this to the package.json file, as we are trying to stop using Bower. I believe all existing dependencies have already been moved, too!

spectral-workbench.js/package.json

Line 27 in 5b176c5

"d3": "~3.5.17",

jywarren · 2020-09-17T16:57:06Z

examples/new-capture/index.html

+  <script src="../../node_modules/flot/jquery.flot.crosshair.js"></script>
+  <script src="../../node_modules/flot/jquery.flot.threshold.js"></script>
+  <script src="../../node_modules/bootstrap/dist/js/bootstrap.min.js"></script>
+  <script src="../../bower_components/jquery.steps/build/jquery.steps.min.js"></script>


Yes - here, and below, all instances of bower_components should be changed to node_modules and all dependencies but jquery.steps have been transferred there already. That should resolve the last error on the final "plotting" page of the new demo!

TonyIversonLin · 2020-09-17T17:20:30Z

If you open the original index.html file, it has a hyper link to the new demo site. Also please do another npm install and bower install before running the new demo page. Thanks.

…

On Thu, Sep 17, 2020, 12:37 PM Jeffrey Warren ***@***.***> wrote: ***@***.**** commented on this pull request. ------------------------------ In examples/new-capture/index_copy.html <#175 (comment)> : > @@ -0,0 +1,363 @@ +<!DOCTYPE html> Just wondering how this is distinct from index.html? Thank you! — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#175 (review)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AEX5LH3NKMVLSEKY6JCM2N3SGI3MNANCNFSM4QZYQAIQ> .

jywarren · 2020-09-24T00:49:41Z

Something's wrong with d3... but in jasmine!

   - can export SVG...X can export SVG
     TypeError: undefined is not an object (evaluating 'd3.select('svg').data()[0][index].id') in file:///home/travis/build/publiclab/spectral-workbench.js/dist/spectral-workbench.js (line 5374) (1)

jywarren · 2020-09-24T00:51:00Z

Gruntfile.js

-                'bower_components/nvd3/build/nv.d3.js',
-                'bower_components/bootstrap-css/js/bootstrap.min.js',
-                'bower_components/moment/moment.js',
+                'node_modules/d3/d3.js',


Could something be wrong with this line, for jasmine?

ah yes! I think d3 would be in a dist folder... checking...

Hmm. <script src="../../node_modules/d3/d3.js" type="text/javascript"></script> indicates not...

jywarren · 2020-09-24T00:55:35Z

Oh hmm, a bunch of Jasmine is broken:


     TypeError: undefined is not an object (evaluating 'd3.select('svg').data()[0][index].id') in file:///home/travis/build/publiclab/spectral-workbench.js/dist/spectral-workbench.js (line 5374) (1)
   - can copy calibrations with copyCalibration()...X can copy calibrations with copyCalibration()
     TypeError: undefined is not an object (evaluating 'graph.datum') in file:///home/travis/build/publiclab/spectral-workbench.js/spec/javascripts/api_spec.js (line 128) (1)
   - generates copyCalibration callback...X generates copyCalibration callback
     Expected spy success to have been called. (1)
   - can transform graph data...X can transform graph data
     TypeError: undefined is not an object (evaluating 'graph.datum') in file:///home/travis/build/publiclab/spectral-workbench.js/spec/javascripts/api_spec.js (line 156) (1)
   - can limit graphed data to wavelength range...X can limit graphed data to wavelength range
     TypeError: undefined is not an object (evaluating 'graph.datum') in file:///home/travis/build/publiclab/spectral-workbench.js/spec/javascripts/api_spec.js (line 168) (1)
   - can blend graphed spectrum data with that of another spectrum...X can blend graphed spectrum data with that of another spectrum

jywarren · 2020-09-29T16:13:17Z

OK, this is odd. But regarding:

 TypeError: undefined is not an object (evaluating 'graph.datum') in file:///home/travis/build/publiclab/spectral-workbench.js/spec/javascripts/api_spec.js (line 128) (1)

That graph object is defined in the previous test:

spectral-workbench.js/spec/javascripts/api_spec.js

Line 103 in 8d0526d

graph = new SpectralWorkbench.Graph({

Is there an order of execution issue happening? No, i think perhaps the SVG test failure is causing the graph object not to be initialized. Let's focus on the SVG test.

jywarren · 2020-09-29T16:50:00Z

OK, reproduced it by going to the graph example at /examples/index.html, copying /spec/javascript/fixtures/spectrum-9.json to /spectrums/9.json (so the aync fetch of the data works), then running:

graph2 = new SpectralWorkbench.Graph({
      spectrum_id: 9,
      calibrated: true,
      onImageComplete: function() { done(); }
    });

Here's the full trace:

Uncaught TypeError: Cannot read property 'id' of undefined
    at SVGGElement.<anonymous> (spectral-workbench.js:5374)
    at SVGGElement.o (d3.min.js:1)
    at d3.min.js:3
    at Y (d3.min.js:1)
    at Array.Co.each (d3.min.js:3)
    at Array.Co.attr (d3.min.js:3)
    at _graph.load (spectral-workbench.js:5373)
    at Object.success (spectral-workbench.js:127)
    at c (jquery.min.js:2)
    at Object.fireWith [as resolveWith] (jquery.min.js:2)

jywarren · 2020-09-29T16:58:10Z

OK - this is a known error: #165 re: d3 v3.5 vs. 3.4.

Don't know why it's just suddenly showing up on Travis. Let's downgrade again and see.

jywarren · 2020-09-30T00:59:08Z

Trying to bump d3 to v3.5.4 because of #180 (comment)

graph demo works:

new capture demo works:

I think we could do this... but will nvd3 work?

jywarren · 2020-09-30T01:03:58Z

Aha ok, so: noting "GET /node_modules/jquery.steps/build/jquery.steps.min.js" Error (404): "Not found"

and, back to the original error:


     TypeError: undefined is not an object (evaluating 'd3.select('svg').data()[0][index].id') in file:///home/travis/build/publiclab/spectral-workbench.js/dist/spectral-workbench.js (line 5374) (1)
   - can copy calibrations with copyCalibration()...X can copy calibrations with copyCalibration()
     TypeError: undefined is not an object (evaluating 'graph.datum') in file:///home/travis/build/publiclab/spectral-workbench.js/spec/javascripts/api_spec.js (line 128) (1)

jywarren · 2020-10-06T16:12:30Z

OK so we have trouble with d3 v3.5 - the known error in our codebase: #165

But v3.4 had caused us trouble with contextify -- specifically we bumped to v3.5.4 to resolve contextify issues in
02c1adc

So we are left with:

solve the known error with TypeError: undefined is not an object (evaluating 'd3.select('svg').data()[0][index].id') in file:///home/travis/build/publiclab/spectral-workbench.js/dist/spectral-workbench.js (line 5374) (1)
find a post-v3.5.4 version that does not have that error - that uses the correct contextify ...?

jywarren · 2020-11-14T21:44:20Z

So, noting that this error would occur a LOT in this file:

spectral-workbench.js/src/SpectralWorkbench.Graph.js

Line 234 in 7b52a77

    
           var id = d3.select('svg').data()[0][index].id; // this is the real d3 DOM-stored data

because this segment causing it is repeatedly called in that file:

var id = d3.select('svg').data()[0][index].id; // this is the real d3 DOM-stored data

The error cited above was for the first instance. So, we can dig into exactly what is not an object but probably there's something wrong with either the DOM or how we're accessing it.

Actually given Cannot read property 'id' of undefined i guess it's d3.select('svg').data()[0][index] that is not an object. Let's see what it is or if the returned format has changed?

jywarren · 2020-11-14T21:54:20Z

Noting the error occurs in client side of the graph demo!

whoa! OK, here are the values:

d3.select('svg').data()[0]
> (4) [{…}, {…}, {…}, {…}]
0: {values: Array(21), key: "undefined (average)", color: "#444", id: undefined}
1: {values: Array(21), key: "undefined (R)", color: "rgba(255,0,0,0.2)"}
2: {values: Array(21), key: "undefined (G)", color: "rgba(0,255,0,0.2)"}
3: {values: Array(21), key: "undefined (B)", color: "rgba(0,0,255,0.2)"}
length: 4__proto__: Array(0)

d3.select('svg').data()[0][0].id
> undefined

Something must be going wrong here while inserting this data:

spectral-workbench.js/src/SpectralWorkbench.Graph.js

Lines 218 to 220 in 7b52a77

    
           _graph.data = d3.select(_graph.selector + ' svg')  //Select the <svg> element you want to render the chart in.    
        
                           .datum(datum.d3)   //Populate the <svg> element with chart data 
        
                           .call(_graph.chart)         //Finally, render the chart!

OK, yes, it's when we call graph.datum.d3():

graph.datum.d3()
> (4) [{…}, {…}, {…}, {…}]
0: {values: Array(21), key: "undefined (average)", color: "#444", id: undefined}
1: {values: Array(21), key: "undefined (R)", color: "rgba(255,0,0,0.2)"}
2: {values: Array(21), key: "undefined (G)", color: "rgba(0,255,0,0.2)"}
3: {values: Array(21), key: "undefined (B)", color: "rgba(0,0,255,0.2)"}

jywarren · 2020-11-14T22:26:52Z

Hmm, maybe the two errors are different, Cannot read property 'id' of undefined vs TypeError: undefined is not an object (evaluating 'd3.select('svg').data()[0][index].id') -- but they seem to affect almost the same segment of code.

jywarren · 2020-11-14T23:54:42Z

Solved TypeError: undefined is not an object (evaluating 'd3.select('svg').data()[0][index].id') in file:///home/travis/build/publiclab/spectral-workbench.js/dist/spectral-workbench.js (line 5374) (1) !!

Now we have one more error -- all these are returning incorrectly... could be related:

   X should initialize line styling and table rows properly
     Expected 1 to be 2. (1)
     Expected 1 to be 2. (2)
     Expected 1 to be 2. (3)
     Expected 1 to be 2. (4)

spectral-workbench.js/spec/javascripts/set_spec.js

Lines 54 to 63 in 7b52a77

    
           it("should initialize line styling and table rows properly", function() { 
        
             expect(graph).toBeDefined(); 
        
             expect(graph.datum).toBeDefined(); 
        
             // you can't do jquery hasClass() to SVGs, but d3 can do classed(); 
        
             expect(d3.selectAll('g#spectrum-line-1')[0].length).toBe(2); // in main graph and in zoom graph 
        
             expect(d3.selectAll('g#spectrum-line-2')[0].length).toBe(2); // in main graph and in zoom graph 
        
             expect(d3.selectAll('g#spectrum-line-4')[0].length).toBe(2); // in main graph and in zoom graph 
        
             expect(d3.selectAll('g#spectrum-line-6')[0].length).toBe(2); // in main graph and in zoom graph

jywarren · 2020-11-15T03:00:14Z

No more test failures!

Just:

101 specs in 14.061s.
>> 0 failures
Fatal error: Callback must be a function�
The command "grunt jasmine" exited with 3.

jywarren · 2020-11-15T03:02:59Z

Related to node 10, switched to 12! gruntjs/grunt-contrib-jasmine#266

jywarren · 2020-11-15T03:31:58Z

YES!!!! All done! Merging! Thank you!!!!

welcome · 2020-11-15T03:32:06Z

Congrats on merging your first pull request! 🙌🎉⚡️
Your code will likely be published to https://spectralworkbench.org in the next few days.
In the meantime, can you tell us your Twitter handle so we can thank you properly?
Now that you've completed this, you can help someone else take their first step!
See: Public Lab's coding community!

TonyIversonLin and others added 18 commits August 28, 2020 09:44

CISC 699 initial commit

f0635b3

package Initialization

2d26192

new router of the new demo page

de91150

add all files and start to refactor ui

5d6a262

half way

5bed37f

add some comments

de41dcd

finished the capturing step

baef78d

Landing Page inital setup complete

8cac73d

Bring in global stepper toolbar, fix styling

8c13593

merge conflict

fcd418b

Merge pull request #1 from TonyIversonLin/yechao-change

40e1dad

Yechao change

More style update

a9d242e

Wire up the plotting UI

8df8400

disable stepper while in plotting step; add restart button on plottin…

1c8614e

…g step

Added testing framework

95372fb

Merge changes

5a25826

Add in basic test case

d66001e

Remove the temp home.html file

23ebbb0

jywarren reviewed Sep 17, 2020

View reviewed changes

jywarren requested changes Sep 17, 2020

View reviewed changes

jywarren reviewed Sep 17, 2020

View reviewed changes

jywarren changed the title ~~Demo site~~ new demo site with integrated capture and graph/download Sep 17, 2020

jywarren changed the title ~~new demo site with integrated capture and graph/download~~ new demo site with integrated capture and graph/download, plus Cypress tests Sep 17, 2020

jywarren reviewed Sep 24, 2020

View reviewed changes

Update .gitignore

c38e339

jywarren mentioned this pull request Sep 29, 2020

Upgrade d3 or use different graphing library for display (not capture) #180

Open

jywarren added 3 commits September 29, 2020 17:08

revert d3 to v3.4

77ba1f8

remove d3 from package-lock

b05ab44

bump d3 to v3.5.4 to resolve contextify issues

02c1adc

jywarren added 2 commits November 14, 2020 22:31

small fixes to demo

8f87228

fixed basic svg and other errors

12f58d2

fixed!

1260187

node 12

77354ba

update grunt-contrib-jasmine

18e2a39

jywarren merged commit ac970d6 into publiclab:main Nov 15, 2020

This was referenced Nov 15, 2020

Fixing getUsermedia shim #125

Closed

Port "capture" interface into this library from main spectral-workbench project #56

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

new demo site with integrated capture and graph/download, plus Cypress tests #175

new demo site with integrated capture and graph/download, plus Cypress tests #175

TonyIversonLin commented Sep 5, 2020

welcome bot commented Sep 5, 2020

gitpod-io bot commented Sep 5, 2020 •

edited

Loading

jywarren Sep 17, 2020

jywarren left a comment

jywarren Sep 17, 2020

jywarren commented Sep 17, 2020

jywarren commented Sep 17, 2020

jywarren commented Sep 17, 2020

jywarren Sep 17, 2020

jywarren Sep 17, 2020

jywarren Sep 17, 2020

TonyIversonLin commented Sep 17, 2020 via email

jywarren commented Sep 24, 2020 •

edited

Loading

jywarren Sep 24, 2020

jywarren Sep 29, 2020

jywarren Sep 29, 2020

jywarren commented Sep 24, 2020

jywarren commented Sep 29, 2020

jywarren commented Sep 29, 2020

jywarren commented Sep 29, 2020 •

edited

Loading

jywarren commented Sep 30, 2020

jywarren commented Sep 30, 2020

jywarren commented Oct 6, 2020 •

edited

Loading

jywarren commented Nov 14, 2020 •

edited

Loading

jywarren commented Nov 14, 2020 •

edited

Loading

jywarren commented Nov 14, 2020

jywarren commented Nov 14, 2020

jywarren commented Nov 15, 2020

jywarren commented Nov 15, 2020

jywarren commented Nov 15, 2020

welcome bot commented Nov 15, 2020

	capture: {
	src: [
	'examples/capture/detect_sample_row.js',
	'examples/capture/toggleRotation.js',
	'examples/capture/getRow.js',
	'examples/capture/getUserMedia.js'
	],
	dest: 'dist/capture.dist.js'
	}



		<script src="capture.js" type="text/javascript"></script>
		<script src="../../dist/capture.dist.js" type="text/javascript"></script>

new demo site with integrated capture and graph/download, plus Cypress tests #175

new demo site with integrated capture and graph/download, plus Cypress tests #175

Conversation

TonyIversonLin commented Sep 5, 2020

welcome bot commented Sep 5, 2020

gitpod-io bot commented Sep 5, 2020 • edited Loading

Choose a reason for hiding this comment

jywarren left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jywarren commented Sep 17, 2020

jywarren commented Sep 17, 2020

jywarren commented Sep 17, 2020

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

TonyIversonLin commented Sep 17, 2020 via email

jywarren commented Sep 24, 2020 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jywarren commented Sep 24, 2020

jywarren commented Sep 29, 2020

jywarren commented Sep 29, 2020

jywarren commented Sep 29, 2020 • edited Loading

jywarren commented Sep 30, 2020

jywarren commented Sep 30, 2020

jywarren commented Oct 6, 2020 • edited Loading

jywarren commented Nov 14, 2020 • edited Loading

jywarren commented Nov 14, 2020 • edited Loading

jywarren commented Nov 14, 2020

jywarren commented Nov 14, 2020

jywarren commented Nov 15, 2020

jywarren commented Nov 15, 2020

jywarren commented Nov 15, 2020

welcome bot commented Nov 15, 2020

gitpod-io bot commented Sep 5, 2020 •

edited

Loading

jywarren commented Sep 24, 2020 •

edited

Loading

jywarren commented Sep 29, 2020 •

edited

Loading

jywarren commented Oct 6, 2020 •

edited

Loading

jywarren commented Nov 14, 2020 •

edited

Loading

jywarren commented Nov 14, 2020 •

edited

Loading