Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

How to capture effected image and save it to file? #93

Open
kesha-antonov opened this issue Apr 3, 2017 · 91 comments
Labels

Comments

@kesha-antonov
Copy link
Contributor

@kesha-antonov kesha-antonov commented Apr 3, 2017

Hello!

In v2 we've had captureFrame function.
https://github.com/ProjectSeptemberInc/gl-react-native/blob/cf77646239ee1c9d6002c28cb8864cb65d2de898/src/GLCanvas.js#L55

But there is no such function in "v3", right? Or I can save image with capture() somehow?

Please help.

I've asked something similar in "v2" repo: gre/gl-react-native-v2#96

question

library version: gl-react/next + gl-react-native/next

gl-react@3.0.0-alpha.6
gl-react-native@3.0.0-alpha.6

Expected behavior

Expected to find some simple function to call and save effected image to file in cache directory

refs.surface.captureFrame({
       type: 'jpg'
       format: 'file'
       quality: 1
       filePath: "#{RNFS.DocumentDirectoryPath}/#{new Date().getTime()}.jpg"
}).then((newImageUri) =>
 ....

Actual behavior

this.refs.surface.capture()

returns array of pixels (right?)

@gre

This comment has been minimized.

Copy link
Owner

@gre gre commented Apr 4, 2017

That's a good point, this function don't exist yet and I should figure out the best way to add it back.
I need to think a bit what would be the best way for that, but what you can do at the moment is capture() that should returns a "ndarray" which is basically an object with a data (Uint32array of pixels) and a shape that your image [width,height,4]. But it would not be trivial nor efficient to try to re-encode that to image on JS thread. So I think I need to add something on the implementation.

The web version have captureAsDataURL and captureAsBlob because it's easier to solve that as there are Web API to do that (canvas.toDataURL & canvas.toBlob).

@gre

This comment has been minimized.

Copy link
Owner

@gre gre commented Apr 4, 2017

a workaround might be to try to use https://github.com/gre/react-native-view-shot for now (I think it only works in iOS tho – regarding a GL view)

@kesha-antonov

This comment has been minimized.

Copy link
Contributor Author

@kesha-antonov kesha-antonov commented Apr 4, 2017

Thanks!
I'll try https://github.com/gre/react-native-view-shot
It can work for me if that package can save image in a good quality (for photos)
2000x1000 px for example

@gre

This comment has been minimized.

Copy link
Owner

@gre gre commented Apr 4, 2017

actually i'm afraid there is also this issue opened and I can't figure out why :'( gre/react-native-view-shot#50

@kesha-antonov

This comment has been minimized.

Copy link
Contributor Author

@kesha-antonov kesha-antonov commented Apr 10, 2017

@gre What do you think about this issue?
Can you help with it or I can make same changes (9376314) in gl-react-native v2 and use it to capture effected image for now.

@gre

This comment has been minimized.

Copy link
Owner

@gre gre commented Apr 10, 2017

gl-react-native v2 don't have the issue, I think the pixel ratio is already correct in v2.

@gre

This comment has been minimized.

Copy link
Owner

@gre gre commented Apr 10, 2017

I think the issue is solvable but I'm not sure the best way to implement. I'm trying to think a nice way to solve it generically and minimize the implementation involved, but probably it's not really possible.. Anyway, basically it will require some more code on both iOS and Android modules.

@kesha-antonov

This comment has been minimized.

Copy link
Contributor Author

@kesha-antonov kesha-antonov commented Apr 10, 2017

@gre Maybe I'm mistaken but v2 does have "blurred image" issue.
That's why I switched to v3 with hope to not find that here.

@gre

This comment has been minimized.

Copy link
Owner

@gre gre commented Apr 10, 2017

Did setting a pixelRatio helped? in v2 there were this concept of pixelRatio – e.g <Surface pixelRatio={2} ...>

@gre

This comment has been minimized.

Copy link
Owner

@gre gre commented Apr 10, 2017

(which by default was supposed to be set to the device pixel ratio size..)

@kesha-antonov

This comment has been minimized.

Copy link
Contributor Author

@kesha-antonov kesha-antonov commented Apr 10, 2017

Oh. Missed that.
I'll try. Thanks!

Anyway if you'll have time to implement "capture" in v3 it would really-really cool! Since I would like to use gl-react to effect Videos too and v2 will not have new features I guess it's better to use v3.

@kesha-antonov

This comment has been minimized.

Copy link
Contributor Author

@kesha-antonov kesha-antonov commented Apr 10, 2017

@gre

This comment has been minimized.

Copy link
Owner

@gre gre commented Apr 10, 2017

yeah, I guess it could even be a generic library that you send the native array from JS and convert it back.

but maybe it would be more performant if it was made as a method from inside EXGLView. i'm not sure how easy it is.

@gre

This comment has been minimized.

Copy link
Owner

@gre gre commented Apr 10, 2017

basically I imagine something like this:

JS side, call a function like EXGLView.capture(viewTag) -> EXGLViewManager.capture static method will resolve viewTag to a EXGLView and call capture on it -> EXGLView.capture needs to run something on the correct thread. I'm not exactly sure what to run (gl.readPixels is meant for JS, maybe there is a smarter way to get pixels directly). also i'm not sure what's the correct thread for that. and how to get it back to JS. (where does it save, basically)

Ideally it could take the same options of that lib https://github.com/gre/react-native-view-shot#takesnapshotview-options

@kesha-antonov

This comment has been minimized.

Copy link
Contributor Author

@kesha-antonov kesha-antonov commented Apr 21, 2017

Hello @gre !
Have you had time to experiment with EXGLView?

@gre

This comment has been minimized.

Copy link
Owner

@gre gre commented May 7, 2017

just want to tell you I did not forget about this task, in term of priority scheduling i'm waiting the release of an incoming version of EXGLView that will allow to do more things around texture & stuff.

@kesha-antonov

This comment has been minimized.

Copy link
Contributor Author

@kesha-antonov kesha-antonov commented May 7, 2017

Hey! Thanks!

Yeah, it would be cool!

@kesha-antonov

This comment has been minimized.

Copy link
Contributor Author

@kesha-antonov kesha-antonov commented May 11, 2017

Hey @gre

What's the best way to try do it with objective-c/java ?

  1. Make function "capture" in gl-react-native
  2. Call from function "1" this function https://github.com/gre/gl-react/blob/master/packages/gl-react/src/Node.js#L547
  3. Call native libs for ios/android and send them array from "2"
  4. Convert byte array of pixels to image and save image in temp
  5. Return image uri

Is this correct?
I want to try it tomorrow

@kesha-antonov

This comment has been minimized.

Copy link
Contributor Author

@kesha-antonov kesha-antonov commented May 12, 2017

JS side, call a function like EXGLView.capture(viewTag) -> EXGLViewManager.capture static method will resolve viewTag to a EXGLView

What is viewTag ?

@kesha-antonov

This comment has been minimized.

Copy link
Contributor Author

@kesha-antonov kesha-antonov commented May 12, 2017

Ok. I can write "save pixels to new image". But first I'm trying to understand how to get "viewTag" and how to get EXGLView from it.

I'm thinking of putting UUID here https://github.com/gre/gl-react/blob/master/packages/gl-react-native/android/src/main/java/fr/greweb/rngl/EXGLView.java#L44
Then return it in "onSurfaceCreate"

Then call EXGLView.capture(viewTag) -> EXGLViewManager.capture

Is it correct? I'm just guessing here

@kesha-antonov

This comment has been minimized.

Copy link
Contributor Author

@kesha-antonov kesha-antonov commented May 12, 2017

Ok. I think viewTag is this.refs.surface.gl or this.refs.surface.glView. Correct ?

@kesha-antonov

This comment has been minimized.

Copy link
Contributor Author

@kesha-antonov kesha-antonov commented May 12, 2017

What if we will take this.refs.surface.gl and call native code:

  1. EXGLView.js: EXGLView.captureFrame(0, 0, 1000, 1000, this.refs.surface.gl)
  2. -> EXGLViewManager.java: createBitmapFromGLSurface(0, 0, 1000, 1000, gl)

?

@kesha-antonov

This comment has been minimized.

Copy link
Contributor Author

@kesha-antonov kesha-antonov commented May 12, 2017

Ok. I'm started working on it:

master...kesha-antonov:capture_frame_from_gl

But I'm not sure how to pass or resolve gl here.

Do you have some advices?

@gre

This comment has been minimized.

Copy link
Owner

@gre gre commented May 12, 2017

maybe see how i did it in https://github.com/gre/react-native-view-shot ?

@kesha-antonov

This comment has been minimized.

Copy link
Contributor Author

@kesha-antonov kesha-antonov commented May 12, 2017

Cool! Thanks. Will try that

@kesha-antonov

This comment has been minimized.

Copy link
Contributor Author

@kesha-antonov kesha-antonov commented May 12, 2017

Ok. I've experimented a little bit more in another brunch but I'm stuck now because of lack of knowledge
master...kesha-antonov:capture_frame_from_gl-1

Will back here later

@kesha-antonov

This comment has been minimized.

Copy link
Contributor Author

@kesha-antonov kesha-antonov commented May 12, 2017

Yes, this is what i'm trying to achieve. Maybe we can port that function in v3?

@kesha-antonov

This comment has been minimized.

Copy link
Contributor Author

@kesha-antonov kesha-antonov commented May 12, 2017

Trying to do capture with https://github.com/gre/react-native-view-shot : black screen

Trying do capture with v2: running out of memory for images 3000x3000
gre/gl-react-native-v2#151

@kesha-antonov

This comment has been minimized.

Copy link
Contributor Author

@kesha-antonov kesha-antonov commented Mar 31, 2018

Hello! Any news?
Almost year passed. @gre Do you have plans to work on this?

@gre

This comment has been minimized.

Copy link
Owner

@gre gre commented Apr 1, 2018

I'll discuss with expo folks on how they intend to solve it in Expo and see what we can do to backport to react-native-webgl. also Expo have a new version of GLView, backed by opengl 3, so I might look at this as well. but it's tricky to maintain react-native-webgl on my own , with limited time

@kesha-antonov

This comment has been minimized.

Copy link
Contributor Author

@kesha-antonov kesha-antonov commented Apr 1, 2018

@gre Thanks for taking your time on this!

@gre

This comment has been minimized.

Copy link
Owner

@gre gre commented May 8, 2018

BTW expo have now a solution for this https://docs.expo.io/versions/v27.0.0/sdk/gl-view#takesnapshotasyncoptions and when we'll figure out how to merge it back to react-native-webgl it will benefit for all ;)

@kytwb

This comment has been minimized.

Copy link

@kytwb kytwb commented May 9, 2018

@gre thanks for the update!

@moeedshahid

This comment has been minimized.

Copy link

@moeedshahid moeedshahid commented May 9, 2018

@kesha-antonov

This comment has been minimized.

Copy link
Contributor Author

@kesha-antonov kesha-antonov commented May 11, 2018

@moeedshahid You can do this only in v2, right? You can see same code in the first comment

@kesha-antonov

This comment has been minimized.

Copy link
Contributor Author

@kesha-antonov kesha-antonov commented May 11, 2018

@gre Thanks for the update! Still waiting for this 🔥

@moeedshahid

This comment has been minimized.

Copy link

@moeedshahid moeedshahid commented May 11, 2018

@kesha-antonov yes i have a lot trouble configuring it with v3 so i downgraded to v2 !

@Jose4gg

This comment has been minimized.

Copy link

@Jose4gg Jose4gg commented May 22, 2018

@gre Do you have an ETA for this feature?

@ParhamZare

This comment has been minimized.

Copy link

@ParhamZare ParhamZare commented May 28, 2018

Hi
Can Apply Filter without Surface And GLIImage like pass image base64 , apply filter and return base64 filtered image ? because the crop function have problem decrease image quality
thanks

@yedeba

This comment has been minimized.

Copy link

@yedeba yedeba commented Jun 22, 2018

@kesha-antonov Hi, please, any example for capture() method in expo?.
I'm not working.
Thanks.

@meinto

This comment has been minimized.

Copy link

@meinto meinto commented Jun 24, 2018

Did anyone implement it for react-native by now? :)

@yedeba

This comment has been minimized.

Copy link

@yedeba yedeba commented Jun 25, 2018

How can I access to capture method declared in the GLViewNative class?
capture = ( opt: * ): Promise<{ uri: string, localUri: string, width: number, height: number }> => { const { ref } = this; if (!ref) return Promise.reject(new Error("glView is unmounted")); return ref.takeSnapshotAsync(opt); };

@yedeba

This comment has been minimized.

Copy link

@yedeba yedeba commented Jun 26, 2018

I already found out. In case someone helps:
let result = await this.refs.surface.glView.capture();

@JinHoSo

This comment has been minimized.

Copy link

@JinHoSo JinHoSo commented Aug 29, 2018

@gre Do you have an ETA for this feature?

@alidawud

This comment has been minimized.

Copy link

@alidawud alidawud commented Sep 3, 2018

Any updates?

@ankhanguit

This comment has been minimized.

Copy link

@ankhanguit ankhanguit commented Sep 10, 2018

Awsome @yedeba, i have used your way. it is successful.
For who using Expo, this is my code:

import { Surface } from "gl-react-expo";

<Surface 
            ref={view => (this.imageFilter = view)} 
            style={styles.photoTakenSection}>
            <ImagePreview picture={picture} selected={filterSelected}/>
</Surface>

_uploadPhoto = async () => {
let pictureSave = await this.imageFilter.glView.capture();
)

The result will have uri of picture, and then you can use it to display or upload on server.

@yedeba

This comment has been minimized.

Copy link

@yedeba yedeba commented Sep 10, 2018

👍👍👍

@gre

This comment has been minimized.

Copy link
Owner

@gre gre commented Sep 10, 2018

great 😝

@dorthwein

This comment has been minimized.

Copy link

@dorthwein dorthwein commented Sep 14, 2018

Is there a non-expo solution for this? :(

@gre

This comment has been minimized.

Copy link
Owner

@gre gre commented Sep 14, 2018

unfortunately for now this feature is only built-in in Expo and not yet backported to react-native-webgl

@elfstyle

This comment has been minimized.

Copy link

@elfstyle elfstyle commented Sep 22, 2018

Hi! this feature worked on Tuesday but on Friday it started to return black pics. I even made test blank project for testing, but result was the same. What changed?

"gl-react-expo": "^3.16.3",
"gl-react-image": "^3.1.1",

import React from 'react';
import {
  CameraRoll,
} from 'react-native';
import { Surface } from 'gl-react-expo'
import GLImage from 'gl-react-image'

export default class HomeScreen extends React.Component {

  handleOnLoad = async () => {
    const result = await this.SurfaceRef.glView.capture();
    console.log(result);
    CameraRoll.saveToCameraRoll(result.uri);
    console.log('done');
  }

  render() {
    return (
      <Surface
        ref={ref => this.SurfaceRef = ref}
        style={{ width: 300, height: 300 }}
        onLoad={this.handleOnLoad}
      >
        <GLImage
          source={{ uri: "http://i.imgur.com/tCatS2c.jpg" }}
        />
      </Surface>
    );
  }
}
@elfstyle

This comment has been minimized.

Copy link

@elfstyle elfstyle commented Sep 22, 2018

Found out that Surface onLoad called handleOnLoad before render finished, if I call handleOnLoad manually, it works.

@serhiipalash

This comment has been minimized.

Copy link

@serhiipalash serhiipalash commented Sep 23, 2018

@elfstyle try to add preload prop to the Surface #159 (comment)

@elfstyle

This comment has been minimized.

Copy link

@elfstyle elfstyle commented Sep 24, 2018

preload helped 👍

@mopilo

This comment has been minimized.

Copy link

@mopilo mopilo commented Nov 29, 2018

any update on this issue ??

@0plus1

This comment has been minimized.

Copy link

@0plus1 0plus1 commented Dec 20, 2018

Hi @elfstyle , @yedeba , @ankhanguit ,

As experts of capturing using expo, is there a reason why my images are always 2400x2400?
Is there a way to force a size, like with: https://docs.expo.io/versions/v31.0.0/sdk/gl-view#expoglviewtakesnapshotasyncgl-options

additionally I cannot seem to track down any documentation regarding glView.capture() is it a binding to what?

Thanks

@markidm

This comment has been minimized.

Copy link

@markidm markidm commented Jan 3, 2019

Any news on the backporting for non-expo users?

@david-saint

This comment has been minimized.

Copy link

@david-saint david-saint commented Jul 7, 2019

It's like from one issue to the other.
I'm tired. I give up.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
You can’t perform that action at this time.