Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Need to consider GPU copy #8

Closed
ds-hwang opened this issue Mar 4, 2015 · 2 comments
Closed

Need to consider GPU copy #8

ds-hwang opened this issue Mar 4, 2015 · 2 comments

Comments

@ds-hwang
Copy link

ds-hwang commented Mar 4, 2015

Currently Mediastream Image Capture spec forces gpu based video to read back to system memory. It hits performance very badly.

First of all, FrameGrabEvent event inherently return software RGBA memory block. It should return handle to point out video frame.

interface FrameGrabEvent : Event {
    readonly    attribute ImageData imageData;
};

In following example, canvas.getContext('2d').drawImage(imgData, ...) is way inefficient than canvas.getContext('2d').drawImage(videoElement, ...).
If video and canvas is gpu accelerated, canvas.getContext('2d').drawImage(videoElement, ...) can copy gpu texture to gpu texture. but canvas.getContext('2d').drawImage(imgData, ...) requires to read back gpu texture to cpu memory and then upload cpu memory to gpu texture.

   if (captureDevice) {
         frameVar = setInterval(captureDevice.grabFrame().then(processFrame()), 1000);
         }
     }

 function processFrame(e) {
     imgData = e.imageData;
     canvas.width = imgData.width;
     canvas.height = imgData.height;
     canvas.getContext('2d').drawImage(imgData, 0, 0,imgData.width,imgData.height);
     }

in the same sense, it's very inefficient to upload the captured frame to WebGL or WebCL.

@ds-hwang
Copy link
Author

ds-hwang commented Mar 4, 2015

@huningxin ^

@gmandyam
Copy link
Collaborator

Will close this issue. PR in #12 seems to solve. Changing from ImageData to ImageBitMap should address this problem.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants