p5.MediaElement size()

[shiffman]

Filing an issue @lmccart and I just talked about related to #1022. We adjusted the associated canvas size when calling loadPixels() so that the pixel array would match the DOM element size. I'm noticing users confused even when just calling image() to draw the image to the canvas that the size doesn't match what is set with size().

[shiffman]

I'm not sure how this happened but I'm looking at p5.dom.js 0.2.7 and the code change from 317fdd1 no longer seems to be in there. This is causing the pixels not to be resized appropriately when calling size() of a p5.MediaElement.

I believe here line 1679:

if (this.canvas.width !== this.elt.width) {
  this.canvas.width = this.elt.videoWidth;
  this.canvas.height = this.elt.videoHeight;
  this.width = this.canvas.width;
  this.height = this.canvas.height;
}

should be:

if (this.canvas.width !== this.elt.width) {
  this.canvas.width = this.elt.width;
  this.canvas.height = this.elt.height;
  this.width = this.canvas.width;
  this.height = this.canvas.height;
}

Or maybe something should be happening when size() is called that makes this work with the existing code? I'm not sure but it is broken again. Happy to pull request the above, but perhaps some other things changed and I'm missing something.

[lmccart]

thanks for catching this! i think there was something tricky with this one... let me look into it tomorrow. what would be super helpful is a unit test for this. i think it could just check the size of elements or a test a few pixels in the pixels array.

[lmccart]

very weird, i can't tell how this got lost but you're right it should def be the second version. i just made the change. thanks!

[shiffman]

This particular fix was reverted due to issue #1274 and the pixels array length does not match the specified size() anymore. You can see how the commit 8022760 changes this.elt.width back to this.elt.videoWidth. We seem to be in an endless cycle of changing this code back and forth. How do we best resolve both of these issues?

I am making a video about this question and when I upload that video I will post a link here.

Just to be clear again, the code that resolves this issue is:

if (this.canvas.width !== this.elt.width) {
  this.canvas.width = this.elt.width;
  this.canvas.height = this.elt.height;
  this.width = this.canvas.width;
  this.height = this.canvas.height;
}

The code that resolves #1274 is:

if (this.canvas.width !== this.elt.width) {
  this.canvas.width = this.elt.videoWidth;
  this.canvas.height = this.elt.videoHeight;
  this.width = this.canvas.width;
  this.height = this.canvas.height;
}

What code would resolve both?

[shiffman]

Here is the video where I filed this issue.

https://youtu.be/6t9m1ePK_-8

[lmccart]

ahhh thank you for tracking down why this part always seems to be broken! and the video is amazing! i don't think this should be too hard to fix, i think the problem previously was just that we didn't realize there were two conflicting cases we kept switching back and forth between. i'll take a look at this in the next few days.

[lmccart]

@Craigson are you still on this one, or @shiffman mentioned you were shifting over to processing work for a while? I can take this one over if you are busy right now, just let me know.

[Craigson]

Hi Lauren,

Sorry this mail slipped by me.. I haven't found a solution yet that solves
both issues, everything I've tried causes the other to break. I plan to
help Dan out with some stuff, but I've had to delay it to try make up
ground on thesis. I was going to try and get back to it this week, but if
it's time sensitive I'm happy to let it go.

On Tue, Apr 12, 2016 at 10:26 AM, Lauren McCarthy notifications@github.com
wrote:

@Craigson https://github.com/Craigson are you still on this one, or
@shiffman https://github.com/shiffman mentioned you were shifting over
to processing work for a while? I can take this one over if you are busy
right now, just let me know.

—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
#1079 (comment)

Craig Pickard
M.P.S Candidate in the Interactive Telecommunications Program, NYU
{ http://craigpickard.net }
{ 929 240 4478 }

[lmccart]

Ok, I think I've fixed this. The fix was to update createCapture() to make sure that this.elt.width and this.elt.height is properly set when the stream is loaded. This allows us to use the correct code inside loadPixels():

if (this.canvas.width !== this.elt.width) {
  this.canvas.width = this.elt.width;
  this.canvas.height = this.elt.height;
  this.width = this.canvas.width;
  this.height = this.canvas.height;
}

Both this issue and #1274 should work properly now.

These were the two test cases I was using:

for issue #1079:

var capture;

function setup() {
  createCanvas(640, 480);
  capture = createCapture(VIDEO);
  capture.size(320, 240);
}

function mousePressed() {
  image(capture, 0, 0);
}

and for issue #1274:

var capture;

function setup() {
  createCanvas(640, 480);
  capture = createCapture(VIDEO);
}

function mousePressed() {
  var img = capture.get(0, 0, width, height);
  image(img, 0, 0, width, height);
} 

please let me know if i missed anything!

[shiffman]

Hooray! I promised to make a follow-up video about this so will do so and test!