RFC: Snapshot API Proposal #9855

Closed
lelandrichardson opened this Issue Sep 12, 2016 · 10 comments

Comments

Projects
None yet
7 participants
@lelandrichardson
Collaborator

lelandrichardson commented Sep 12, 2016

Motivation

"Snapshotting" is a method commonly used in native app development to provide an efficient way to visually "clone" view hierarchies in a side-effect free way. Having these capabilities allows for more nontrivial animations, especially those that cross boundaries of react component hierarchies, and perhaps need multiple visual copies of the element in order to produce the desired effect.

Potential Use Cases

  • Reflection View
  • Shared Element Transitions
  • "Explosion" Effect
  • Screen Chooser
  • ...
  • Profit?

Interface

SnapshotRef

A SnapshotRef is an opaque data structure that acts as a light-weight and serializable identifier for a snapshot stored in the native context. Because
there is no way to practically inspect this data structure for an actual image
in JavaScript, we are allowing ourselves to create and use these references synchronously from the JavaScript thread. The only thing you can do with a SnapshotRef that is of use, is pass it to an <Image /> component as a source prop.

Additionally, the developer is expected to remember to dispose of a SnapshotRef, or else the actual snapshot will be held in memory outside of the JS context.

Snapshot: React.Component

Static Methods

Snapshot.create(node: NodeHandle) => SnapshotRef

Snapshot.dispose(ref: SnapshotRef) => void

Snapshot.update(ref: SnapshotRef) => SnapshotRef returns a new SnapshotRef representing an freshly-taken snapshot of the same view the passed in SnapshotRef was a snapshot of.

Notes:

  1. If Snapshot.update gets passed a snapshot of a view that no longer exists, the snapshot won't be able to be taken. We could warn the user, but we can't throw because we wouldn't know synchronously (unless we can ask whether a node exists in JS? Hmmm.... Maybe we can?).
  2. It's unclear if Snapshot.update should dispose of the passed in ref or not. There are definitely cases where we wouldn't want to, but I wonder if those cases should be opt out?

Component Props

...View.propTypes
snapshot: SnapshotRef

Notes:

  1. There are a lot of cases where we probably want to render a snapshot alongside a currently-being-rendered react element, and we don't have the nodeHandle yet as a result. We may want to build Snapshot in such a way that it is resilient to having null or something passed in as the node handle?
  2. We will also want to export a Snapshot.Animated component or something.

Usage

Mirror / Clone Effect:

import { Snapshot } from 'react-native';

class SomeComponentWithMirror extends React.Component {
  render() {
    const mirrorStyle = {
      transform: [ ... ],
    };
    return (
      <View>
       <SomeComponent ref="foo" ... />
       <Snapshot 
         snapshot={Snapshot.create(this.refs.foo)} 
         style={mirrorStyle}
       />
      </View>
    );
  }
}

Note: what's nice about this is you could actually make a generic component that did this and didn't know anything about the component it was rendering.

Shared Element Transitions:
The code here is a bit more complicated for a simple example, but the basic idea would be that you would annotate views in one screen and another screen and pair them together. You would take snapshots of them all, and then create a new "animation container" view that you'd put all of the snapshot views in, positioned absolutely... and then animate them manually.

Implementation

This API is taking advantage of highly efficient snapshotting APIs that exist on both the iOS and Android platforms. On iOS, we will be using the snapshotViewAfterScreenUpdates method, and on Android we will be using the View::getDrawingCache method.

JavaScript:

// Snapshot.js
const { SnapshotModule } = require('NativeModules');

const createRef: nodeHandle => ({
  __isSnapshot: true,
  __nodeHandle: nodeHandle,
  __snapshotId: uuid(),
});

class Snapshot extends React.Component {
  static create: (nodeHandle) => {
    const ref = createRef(nodeHandle);
    SnapshotModule.create(ref.__nodeHandle, ref.__snapshotId);
    return ref;
    };
  static dispose: (ref) => {
    SnapshotModule.dispose(ref.__snapshotId);
  };
  static propTypes: {
    ...View.propTypes,
    snapshot: PropTypes.shape({ 
      __isSnapshot: PropTypes.bool.isRequired,
    }).isRequired,
  };
  render() {
    return <NativeSnapshotView {...this.props } />;
  }
}

module.exports = Snapshot;

Potential additional component that takes a node, and handles lifecycle for you:

const Snapshot = require('Snapshot');

class NodeSnapshot extends React.Component {
  static propTypes: {
    ...View.propTypes,
    node: NodePropType.isRequired,
  };
  constructor(props) {
    super(props);
    this.state = { snapshot: Snapshot.create(props.node) };
  }
  componentWillReceiveProps(props) {
    if (props.node !== this.props.node) {
      Snapshot.dispose(this.state.snapshot);
      this.setState({
        snapshot: Snapshot.create(props.node),
      });
    }
  }
  componentWillUnmount() {
    Snapshot.dispose(this.state.snapshot);
  }
  render() {
    return <NativeSnapshotView {...this.props } snapshot={this.state.snapshot} />;
  }
}
module.exports = NodeSnapshot;

iOS:

struct Snapshot {
  let id: String
  let snapshot: UIView
}

class SnapshotModule: RCTBridgeModule {
  private let snapshotMap: [String: Snapshot]
  func create(nodeHandle: Int, snapshotId: String) {
    if let view = UIManager.getViewByNodeHandle(nodeHandle) {
      let snapshot = view.snapshot(afterUpdates: true)
      snapshotMap[snapshotId] = Snapshot(
        id: snapshotId,
        snapshot: snapshot
      )
      }
  }
  func dispose(snapshotId: String) {
    snapshotMap[snapshotId] = nil
  }
}

class SnapshotViewManager: RCTViewManager {
     // details still a little fuzzy here. But essentially,
  // we will be rendering a view that will be the
  // size of the snapshot, with the snapshot inside of it
}

@brentvatne

This comment has been minimized.

Show comment
Hide comment
Collaborator

brentvatne commented Sep 12, 2016

@janicduplessis

This comment has been minimized.

Show comment
Hide comment
@janicduplessis

janicduplessis Sep 12, 2016

Collaborator

👍 I was thinking if a similar API too. Is there any advantage to expose the lower level api of create/update/dispose snapshot instead of having it managed by the component using lifecycle methods and just passing the ref directly as a prop instead? I guess it would make is possible to display the same snapshot in multiple views but I dont know if its a use case worth adding this complexity.

Collaborator

janicduplessis commented Sep 12, 2016

👍 I was thinking if a similar API too. Is there any advantage to expose the lower level api of create/update/dispose snapshot instead of having it managed by the component using lifecycle methods and just passing the ref directly as a prop instead? I guess it would make is possible to display the same snapshot in multiple views but I dont know if its a use case worth adding this complexity.

@gre

This comment has been minimized.

Show comment
Hide comment
@gre

gre Sep 12, 2016

Contributor

very interesting 👍

the reason I created react-native-view-shot was that UIManager.takeSnapshot was not going to make it to Android (see PR). My current use-case being limited to just "I want to export a rendering to share an image result to social networks".

I really hope we can push your idea to be built in RN, I especially like your dispose & update API proposal. if it can't be, I still imagine it could be possible to implement as a third party?

If you need another use-case: I could definitely imagine it could be a solid API to implement caching of views like OpenGL views. For instance to implement something like https://github.com/gre/gl-react-dom-static-container (in WebGL, it was mandatory to have this, as you are limited in maximum concurrent WebGL contexts)

One question: do you think the update can actually be implemented faster than dispose(old) + create(new) ? Is it just so you don't "leak" images that you have designed the update or do you think it can be implemented in a way that's it's actually faster to keep snapshotting the same view & reusing same image?

cheers

Contributor

gre commented Sep 12, 2016

very interesting 👍

the reason I created react-native-view-shot was that UIManager.takeSnapshot was not going to make it to Android (see PR). My current use-case being limited to just "I want to export a rendering to share an image result to social networks".

I really hope we can push your idea to be built in RN, I especially like your dispose & update API proposal. if it can't be, I still imagine it could be possible to implement as a third party?

If you need another use-case: I could definitely imagine it could be a solid API to implement caching of views like OpenGL views. For instance to implement something like https://github.com/gre/gl-react-dom-static-container (in WebGL, it was mandatory to have this, as you are limited in maximum concurrent WebGL contexts)

One question: do you think the update can actually be implemented faster than dispose(old) + create(new) ? Is it just so you don't "leak" images that you have designed the update or do you think it can be implemented in a way that's it's actually faster to keep snapshotting the same view & reusing same image?

cheers

@lelandrichardson

This comment has been minimized.

Show comment
Hide comment
@lelandrichardson

lelandrichardson Sep 12, 2016

Collaborator

@janicduplessis

Is there any advantage to expose the lower level api of create/update/dispose snapshot instead of having it managed by the component using lifecycle methods and just passing the ref directly as a prop instead?

I thought about this a bit. I think it's a good idea to have a snapshot component that just accepts a node and handles all of the lifecycle for you... but I don't think that it is enough to do just that. For the most part, when you are using snapshots you might want to reuse and unmount/mount etc. and the second we call "dispose", we will never be able to get it back again. It'd be nice if we could come up with a component that handles the most common use cases without people needing to deal with disposal, but i don't think that we can only provide that component.

@gre

if it can't be, I still imagine it could be possible to implement as a third party?

Yes. Nothing here requires us to use react internals... as a result, we can make a 3rd party solution. This RFC is mainly to identify whether or not we want to put it in core, and to discuss refinements on the proposed API.

One question: do you think the update can actually be implemented faster than dispose(old) + create(new) ? Is it just so you don't "leak" images that you have designed the update or do you think it can be implemented in a way that's it's actually faster to keep snapshotting the same view & reusing same image?

The main purpose for the "update" API is to allow a consumer of a SnapshotRef to get an "update" of the original image/view, without needing a reference to the node directly. There is no performance benefit, though.

Collaborator

lelandrichardson commented Sep 12, 2016

@janicduplessis

Is there any advantage to expose the lower level api of create/update/dispose snapshot instead of having it managed by the component using lifecycle methods and just passing the ref directly as a prop instead?

I thought about this a bit. I think it's a good idea to have a snapshot component that just accepts a node and handles all of the lifecycle for you... but I don't think that it is enough to do just that. For the most part, when you are using snapshots you might want to reuse and unmount/mount etc. and the second we call "dispose", we will never be able to get it back again. It'd be nice if we could come up with a component that handles the most common use cases without people needing to deal with disposal, but i don't think that we can only provide that component.

@gre

if it can't be, I still imagine it could be possible to implement as a third party?

Yes. Nothing here requires us to use react internals... as a result, we can make a 3rd party solution. This RFC is mainly to identify whether or not we want to put it in core, and to discuss refinements on the proposed API.

One question: do you think the update can actually be implemented faster than dispose(old) + create(new) ? Is it just so you don't "leak" images that you have designed the update or do you think it can be implemented in a way that's it's actually faster to keep snapshotting the same view & reusing same image?

The main purpose for the "update" API is to allow a consumer of a SnapshotRef to get an "update" of the original image/view, without needing a reference to the node directly. There is no performance benefit, though.

@janicduplessis

This comment has been minimized.

Show comment
Hide comment
@janicduplessis

janicduplessis Sep 12, 2016

Collaborator

@lelandrichardson Sounds good, I guess we could make it so that snapshot prop can accept either a node or a snapshot object.

Collaborator

janicduplessis commented Sep 12, 2016

@lelandrichardson Sounds good, I guess we could make it so that snapshot prop can accept either a node or a snapshot object.

@lelandrichardson

This comment has been minimized.

Show comment
Hide comment
@lelandrichardson

lelandrichardson Sep 12, 2016

Collaborator

@janicduplessis that's how I first wrote up my example... but i thought it made things a bit confusing so i split it out.

Collaborator

lelandrichardson commented Sep 12, 2016

@janicduplessis that's how I first wrote up my example... but i thought it made things a bit confusing so i split it out.

@janicduplessis janicduplessis referenced this issue in expo/ex-navigation Sep 13, 2016

Closed

[WIP] Shared element transitions #83

expbot added a commit to expo/ex-navigation that referenced this issue Sep 30, 2016

Shared element transitions support (experimental! API will change, us…
…e at your own risk)

This implements shared element transitions using react only. This is a work in progress while we figure out the whole API and fix issues / hacks.

The API is based off https://gist.github.com/lelandrichardson/a37baf613fd96ff2b52711dc78094cc2 and work / discussions by @skevy and @lelandrichardson.

Once view snapshot is possible it will be possible to use that too so it is more performant. Related issue: facebook/react-native#9855.

Closes #83

fbshipit-source-id: 7f6fb30
@charpeni

This comment has been minimized.

Show comment
Hide comment
@charpeni

charpeni Nov 14, 2016

Collaborator

@facebook-github-bot label Icebox

Collaborator

charpeni commented Nov 14, 2016

@facebook-github-bot label Icebox

@charpeni

This comment has been minimized.

Show comment
Hide comment
@charpeni

charpeni Nov 14, 2016

Collaborator

Hi there! This issue is being closed because it has been inactive for a while.

But don't worry, it will live on with ProductPains! Check out its new home: https://productpains.com/post/react-native/rfc-snapshot-api-proposal

ProductPains helps the community prioritize the most important issues thanks to its voting feature.
It is easy to use - just login with GitHub.

Also, if this issue is a bug, please consider sending a PR with a fix.
We're a small team and rely on the community for bug fixes of issues that don't affect fb apps.

Collaborator

charpeni commented Nov 14, 2016

Hi there! This issue is being closed because it has been inactive for a while.

But don't worry, it will live on with ProductPains! Check out its new home: https://productpains.com/post/react-native/rfc-snapshot-api-proposal

ProductPains helps the community prioritize the most important issues thanks to its voting feature.
It is easy to use - just login with GitHub.

Also, if this issue is a bug, please consider sending a PR with a fix.
We're a small team and rely on the community for bug fixes of issues that don't affect fb apps.

@charpeni

This comment has been minimized.

Show comment
Hide comment
Collaborator

charpeni commented Nov 14, 2016

@facebook-github-bot

This comment has been minimized.

Show comment
Hide comment
@facebook-github-bot

facebook-github-bot Nov 14, 2016

@charpeni tells me to close this issue. If you think it should still be opened let us know why.

@charpeni tells me to close this issue. If you think it should still be opened let us know why.

@facebook facebook locked as resolved and limited conversation to collaborators May 24, 2018

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.