New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Component ratio support #62

Open
gliechtenstein opened this Issue Jul 12, 2017 · 9 comments

Comments

Projects
None yet
4 participants
@gliechtenstein
Contributor

gliechtenstein commented Jul 12, 2017

There are several places where we can optimize to make Jasonette scroll smoothly.

One of those cases is handling component dimension. To understand why this is so, I need to discuss how this is implemented internally. You can skip this part and jump to the next "proposal" section if you're just curious about the proposed solution.

1. How app layout works

How Jasonette handles layout

All mobile devices implement some sort of "autolayout" features (both iOS and Android). This makes sure components without a dimension attribute will be automatically resized to "just work".

But sometimes the autolayout feature gets confused how exactly to lay them out. This is especially the case when you're loading an image. If you don't set the width and height directly on the image, Jasonette will try to figure it out AFTER the image has loaded by looking at the image file itself.

This normally does a good job but in some cases this resolution is visible and that's what makes it feel "jerky" when scrolling. Currently you can get around this if you use fixed width and height (must set both width and height)

How most native apps handle layout

Also note that this is not a "native vs. non-native" problem. Just take a look at a google search page for https://www.google.com/search?q=ios+scroll+jerky&oq=ios+scroll+&aqs=chrome.0.69i59j69i57j0l4.16386j0j7&sourceid=chrome&ie=UTF-8#q=ios+scroll+stutter This is a very common problem even among native ios developers.

And this is why a lot of apps have uniform components (Instagram used to only allow square images and many apps have fixed media sizes just to not worry about all the complexity that arise by allowing variable component sizes).

And even the apps that do have variable component sizes have to work hard to implement custom optimization logic for their apps. While difficult this is possible if you have a specific business logic because you can build your optimization logic around this assumption.

But the goal of Jasonette is to try to come up with a reusable approach to handling these things, so the optimization needs to be universal and shouldn't depend on a specific business logic.

2. Proposal

We can get around a lot of layout performance issues by:

  1. having a uniform component size
  2. setting the width and height manually

But in many cases we want to have variable sized components AND make it work. Like I mentioned in the previous section, this is not an easy task and there is no silver bullet approach to solving this in one fell swoop, so I think the best approach is to keep improving the most obvious use cases.

One such case is components where we know:

  1. at least either the width or the height
  2. the ratio of width vs. height

Here are some examples:

An image with width of 30% and height that's twice the size of the width:

{
  "type": "image",
  "url": "http://...",
  "style": {
    "width": "30%",
    "ratio": "2"
  }
}

An image with width of 100 and height that's half the size of the width:

{
  "type": "image",
  "url": "http://...",
  "style": {
    "width": "100",
    "ratio": "0.5"
  }
}

Because of the way components are implemented I think we can apply this not just to images but all components, which would improve the performance for this specific case.

If you have better ideas about the JSON syntax, or any other ideas, feel free to share.

@drwasho

This comment has been minimized.

Show comment
Hide comment
@drwasho

drwasho Jul 13, 2017

I really like this @gliechtenstein, I also think that while we're at it (and if it's not too much trouble) we should be looking at image scaling:

{
  "type": "image",
  "url": "http://...",
  "style": {
    "width": "100",
    "ratio": "0.5",
    "scaling": "0.5"
  }
}

Or negative (zoom out):

{
  "type": "image",
  "url": "http://...",
  "style": {
    "width": "100",
    "ratio": "0.5",
    "scaling": "-0.7"
  }
}

drwasho commented Jul 13, 2017

I really like this @gliechtenstein, I also think that while we're at it (and if it's not too much trouble) we should be looking at image scaling:

{
  "type": "image",
  "url": "http://...",
  "style": {
    "width": "100",
    "ratio": "0.5",
    "scaling": "0.5"
  }
}

Or negative (zoom out):

{
  "type": "image",
  "url": "http://...",
  "style": {
    "width": "100",
    "ratio": "0.5",
    "scaling": "-0.7"
  }
}
@gliechtenstein

This comment has been minimized.

Show comment
Hide comment
@gliechtenstein

gliechtenstein Jul 13, 2017

Contributor

@drwasho how does scaling work?

Contributor

gliechtenstein commented Jul 13, 2017

@drwasho how does scaling work?

@drwasho

This comment has been minimized.

Show comment
Hide comment
@drwasho

drwasho Jul 13, 2017

It would analogous to transform: scale(1.3) in CSS, so zooming into or out of an image.

drwasho commented Jul 13, 2017

It would analogous to transform: scale(1.3) in CSS, so zooming into or out of an image.

@jashot7

This comment has been minimized.

Show comment
Hide comment
@jashot7

jashot7 Jul 13, 2017

I like, appreciate and agree upon a focus with either:

  • the width or the height
  • the ratio of width vs. height

jashot7 commented Jul 13, 2017

I like, appreciate and agree upon a focus with either:

  • the width or the height
  • the ratio of width vs. height
@gliechtenstein

This comment has been minimized.

Show comment
Hide comment
@gliechtenstein

gliechtenstein Jul 21, 2017

Contributor

Guys, please check out the ratio branch (Currently only on iOS)

I would appreciate it if someone could try it out and share feedback. I think this will improve rendering for situations mentioned above, but I don't personally have an example where this actually matters.

Contributor

gliechtenstein commented Jul 21, 2017

Guys, please check out the ratio branch (Currently only on iOS)

I would appreciate it if someone could try it out and share feedback. I think this will improve rendering for situations mentioned above, but I don't personally have an example where this actually matters.

@darenr

This comment has been minimized.

Show comment
Hide comment
@darenr

darenr Jul 21, 2017

Contributor

I will give it a try, some initial feedback on syntax:

  • I'm guessing that I can do the ratio calculation in javascript as in:
{
  "type": "image",
  "url": "http://...",
  "style": {
    "width": "100",
    "ratio": "{{return image.actual_width/image.actual_height}}",
  }
}

But it would be useful maybe to also support in the syntax math equations to avoid having to make the ratio calculation because my use case where I do server-side templating is easy to come up with the image ratio is probably atypical - what would be more typical would be someone knowing the image actual dimensions (and not the ratio). My proposal would be supporting in ratio an actual ratio as in:

  "type": "image",
  "url": "http://...",
  "style": {
    "width": "100",
    "ratio": "2:1",
  }
}

Probably that would make it easier to use the feature. Going one small step further, I would suggest since I can never remember anything, to put into the key name what the ratio direction is, as in:

  "type": "image",
  "url": "http://...",
  "style": {
    "width": "100",
    "width_height_ratio": "2:1",
  }
}

As an aside, I found that the jumping around and resizing of images is far less noticeable on Android that iOS, no idea why, just the feedback I've seen.

Do you think this branch will address the timing issue where sometimes images get stretched improperly - ending up with the incorrect aspect ratio? It seems as though that's got to be some kind of timing because it's inconsistent and impossible to reproduce for you.

Contributor

darenr commented Jul 21, 2017

I will give it a try, some initial feedback on syntax:

  • I'm guessing that I can do the ratio calculation in javascript as in:
{
  "type": "image",
  "url": "http://...",
  "style": {
    "width": "100",
    "ratio": "{{return image.actual_width/image.actual_height}}",
  }
}

But it would be useful maybe to also support in the syntax math equations to avoid having to make the ratio calculation because my use case where I do server-side templating is easy to come up with the image ratio is probably atypical - what would be more typical would be someone knowing the image actual dimensions (and not the ratio). My proposal would be supporting in ratio an actual ratio as in:

  "type": "image",
  "url": "http://...",
  "style": {
    "width": "100",
    "ratio": "2:1",
  }
}

Probably that would make it easier to use the feature. Going one small step further, I would suggest since I can never remember anything, to put into the key name what the ratio direction is, as in:

  "type": "image",
  "url": "http://...",
  "style": {
    "width": "100",
    "width_height_ratio": "2:1",
  }
}

As an aside, I found that the jumping around and resizing of images is far less noticeable on Android that iOS, no idea why, just the feedback I've seen.

Do you think this branch will address the timing issue where sometimes images get stretched improperly - ending up with the incorrect aspect ratio? It seems as though that's got to be some kind of timing because it's inconsistent and impossible to reproduce for you.

@gliechtenstein

This comment has been minimized.

Show comment
Hide comment
@gliechtenstein

gliechtenstein Jul 21, 2017

Contributor

@darenr done Jasonette/JASONETTE-iOS@cdb8bd3

You can do "2:1" as well as "1/9"

Do you think this branch will address the timing issue where sometimes images get stretched improperly - ending up with the incorrect aspect ratio?

I am guessing this happens when you only specify just width or just height. Which means if you know the width and the ratio, or the height and the ratio, the stretch wouldn't happen. But this assumes that you know the ratio.

That said, I do think we need to fix that bug even if it happens occasionally. From my experience, a lot of bugs are symptoms of something more fundamental, so if you ever come across a case where this can be replicated with higher chance please share and I'll take a look.

Contributor

gliechtenstein commented Jul 21, 2017

@darenr done Jasonette/JASONETTE-iOS@cdb8bd3

You can do "2:1" as well as "1/9"

Do you think this branch will address the timing issue where sometimes images get stretched improperly - ending up with the incorrect aspect ratio?

I am guessing this happens when you only specify just width or just height. Which means if you know the width and the ratio, or the height and the ratio, the stretch wouldn't happen. But this assumes that you know the ratio.

That said, I do think we need to fix that bug even if it happens occasionally. From my experience, a lot of bugs are symptoms of something more fundamental, so if you ever come across a case where this can be replicated with higher chance please share and I'll take a look.

@gliechtenstein

This comment has been minimized.

Show comment
Hide comment
@gliechtenstein
Contributor

gliechtenstein commented Jul 21, 2017

@gliechtenstein

This comment has been minimized.

Show comment
Hide comment
@gliechtenstein

gliechtenstein Jul 21, 2017

Contributor

BTW @drwasho I took a look at whether it's feasible to add scale as well, and this turns out to be much more involved and require a lot of time, so let's revisit this one later.

Contributor

gliechtenstein commented Jul 21, 2017

BTW @drwasho I took a look at whether it's feasible to add scale as well, and this turns out to be much more involved and require a lot of time, so let's revisit this one later.

gliechtenstein added a commit to Jasonette/JASONETTE-iOS that referenced this issue Jul 25, 2017

Support aspect ratio for components (#244)
* Support aspect ratio for components Jasonette/documentation#62

* More ways of expressing ratio (ex: 2:1,1/3)

* Support ratio for layers

* Remove image prefetching to optimize loading

## Solution

DO NOT prefetch if:
1. height is fixed
2. width and ratio are specified

## What problem it solves

Previously ALL images were prefetched because that was the only way to estimate the height, which is how Jasonette constructs the layout in a robust manner.

This mostly works and does what it's designed to do, but with a side effect:

Since it prefetches EVERY image on the screen, say if you had 2000 images, the view would start prefetching ALL of these images immediately upon load, even those that are not immediately visible because they're outside the viewport. This means you have 2000 concurrent network requests open just for these images.

Normally a view doesn't contain that many images and it works just fine as is, but when we have a lot of images it can congest up the network, sometimes even resulting in a delayed load of the images that DO need to load immediately.

For example the images in the currently visible viewport need to be loaded as soon as possible, but with prefetching this becomes non-deterministic, especially when there are a lot of images. This is why sometimes it feels like an image is loading forever when you open the app.

## Effect

This should have an immediate effect if your view contains a lot of images. More specifically image loading should feel visibly faster (because there's no prefetch and it only loads the images that need to be loaded on-demand)

To get this to work, you need to either set:

1. style.height
2. style.width AND style.ratio
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment