Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[encoding] Bump estimate for segments #454

Merged
merged 1 commit into from
Mar 15, 2024
Merged

Conversation

armansito
Copy link
Collaborator

The segments (and segment counts) buffers contain intermediate data structures that associate lines (post-flatten, post-binning) with tiles. Their overall count depends on the number tile crossings for each line.

These counts are now estimated as follows:

  1. Explict lines with known endpoints are estimated precisely by guessing the worst case rasterization as the sum of the tiling estimate for the x and y coordinates.

  2. Explicit lines with unknown endpoints (e.g. the caps on a stroke) are estimated based on the length of the segment that is near-diagonal.

  3. Flattened curves are estimated by tiling a line whose length is derived based on a conservative estimate of the curve's arc length. The segment count is forced to be at least as large as the Wang's estimate used for the LineSoup count.

This implementation currently has two major shortcomings:

I. Clipping is not taken into account, so the count is overstimated when
layers are present or when geometry lies outside the viewport. There
are ways to address this but they will require some changes:

a) Most of the count accummulation happens at encode time but the
bounds of the viewport are only known at render time. This could
be changed so that most of the accummulation happens at render
time and additional data (like the bounding box of each shape) get
tracked during encoding.

b) It might make sense to track the clip stack to test each shape's
bounding box against the clip geometry while resolving the counts
and use that as input to a heuristic. It is also possible to
discard shapes that lie completely outside the bounds of the clip
geometry. All of this would require additional tracking that
impacts CPU time and memory usage.

II. A rotation that is present in the transform has an impact on tile
crossings. We can precisely account for this for explicit lines with
known endpoints but we have to use a heuristic for curves. The
estimator doesn't track detailed shape data, so a heuristic must be
used when appending a scene fragment. We currently inflate all
segment counts as if they have a 90 degree rotation whenever a
transform should apply.

Overall the segment count tends to be overestimated 3x-10x. There is one known failure mode when the count is underestimated with very small scale factors ("conflation artifacts" test scene).

This PR is based on #436


Some examples:

  • tiger (LineSoup: 2.67x, Segments: 7.13x)
Last frame estimated bump buffer counts:
        Total:                  10385920 bytes (10142.50 KB | 9.90 MB)
        Segment Counts:         281024 elements (2248192 bytes)
        Segments:               281024 elements (6744576 bytes)
        Lines:                  58048 elements (1393152 bytes)

Last frame actual bump buffer counts:
        Total:                  1782528 bytes (1740.75 KB | 1.70 MB)
        Segment Counts:         39441 elements (315528 bytes)
        Segments:               39441 elements (946584 bytes)
        Lines:                  21684 elements (520416 bytes)

  • tricky_strokes (LineSoup: 2.1x, Segments: 2.36x)
Last frame estimated bump buffer counts:
        Total:                  285472 bytes (278.78 KB | 0.27 MB)
        Segment Counts:         6788 elements (54304 bytes)
        Segments:               6788 elements (162912 bytes)
        Lines:                  2844 elements (68256 bytes)

Last frame actual bump buffer counts:
        Total:                  124416 bytes (121.50 KB | 0.12 MB)
        Segment Counts:         2874 elements (22992 bytes)
        Segments:               2874 elements (68976 bytes)
        Lines:                  1352 elements (32448 bytes)

  • cardioid (LineSoup: 2.28x, 1.14x)
Last frame estimated bump buffer counts:
        Total:                  4063672 bytes (3968.43 KB | 3.88 MB)
        Segment Counts:         116549 elements (932392 bytes)
        Segments:               116549 elements (2797176 bytes)
        Lines:                  13921 elements (334104 bytes)

Last frame actual bump buffer counts:
        Total:                  3413672 bytes (3333.66 KB | 3.26 MB)
        Segment Counts:         102433 elements (819464 bytes)
        Segments:               101992 elements (2447808 bytes)
        Lines:                  6100 elements (146400 bytes)

  • longpathdash w/ round caps (LineSoup: 2.27x, Segments: 4.66x)
Last frame estimated bump buffer counts:
        Total:                  530912136 bytes (518468.88 KB | 506.32 MB)
        Segment Counts:         12441912 elements (99535296 bytes)
        Segments:               12441912 elements (298605888 bytes)
        Lines:                  5532123 elements (132770952 bytes)

Last frame actual bump buffer counts:
        Total:                  143974480 bytes (140600.08 KB | 137.30 MB)
        Segment Counts:         2672414 elements (21379312 bytes)
        Segments:               2672414 elements (64137936 bytes)
        Lines:                  2435718 elements (58457232 bytes)

  • mmark 10,000 segments (LineSoup: 1.8x, Segments: 2.33
Last frame estimated bump buffer counts:
        Total:                  28628048 bytes (27957.08 KB | 27.30 MB)
        Segment Counts:         681289 elements (5450312 bytes)
        Segments:               681289 elements (16350936 bytes)
        Lines:                  284450 elements (6826800 bytes)

Last frame actual bump buffer counts:
        Total:                  13146592 bytes (12838.47 KB | 12.54 MB)
        Segment Counts:         292934 elements (2343472 bytes)
        Segments:               292934 elements (7030416 bytes)
        Lines:                  157196 elements (3772704 bytes)

Copy link
Contributor

@raphlinus raphlinus left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks good to me. I am finding myself perhaps a bit nerd-sniped by trying to figure out better estimation heuristics, but I think the right way to approach that is empirical. Of course, the Euler changes will probably require some re-tuning, but I'm pretty hopeful it won't cause the estimations to need to go up substantially. Thanks!

@armansito armansito force-pushed the bump-estimate branch 3 times, most recently from 7bfca34 to ac079c1 Compare March 12, 2024 20:07
Base automatically changed from bump-estimate to main March 12, 2024 20:46
The segments (and segment counts) buffers contain intermediate data
structures that associate lines (post-flatten, post-binning) with tiles.
Their overall count depends on the number tile crossings for each line.

These counts are now estimated as follows:

1. Explict lines with known endpoints are estimated precisely by
   guessing the worst case rasterization of the hypothenuse (i.e. the
   sum of the tiling estimate for the x and y coordinates.

2. Explicit lines with unknown endpoints (e.g. the caps on a stroke)
   are estimated based on the length of the segment.

3. Flattened curves are estimated by tiling a line whose length is
   derived based on a conservative estimate of the curve's arc length.
   The segment count is forced to be at least as large as the Wang's
   estimate used for the LineSoup count.

This implementation currently has two major shortcomings:

I. Clipping is not taken into account, so the count is overstimated when
   layers are present or when geometry lies outside the viewport. There
   are ways to address this but they will require some changes:

   a) Most of the count accummulation happens at encode time but the
      bounds of the viewport are only known at render time. This could
      be changed so that most of the accummulation happens at render
      time and additional data (like the bounding box of each shape) get
      tracked during encoding.

   b) It might make sense to track the clip stack to test each shape's
      bounding box against the clip geometry while resolving the counts
      and use that as input to a heuristic. It is also possible to
      discard shapes that lie completely outside the bounds of the clip
      geometry. All of this would require additional tracking that
      impacts CPU time and memory usage.

II. A rotation that is present in the transform has an impact on tile
    crossings. We can precisely account for this for explicit lines with
    known endpoints but we have to use a heuristic for curves. The
    estimator doesn't track detailed shape data, so a heuristic must be
    used when appending a scene fragment. We currently inflate all
    segment counts as if they have a 90 degree rotation whenever a
    transform should apply.

Overall the segment count tends to be overestimated 3x-10x. There is one
known failure mode when the count is underestimated with _very_ small
scale factors ("conflation artifacts" test scene).
@armansito armansito added this pull request to the merge queue Mar 15, 2024
Merged via the queue into main with commit 829eb2c Mar 15, 2024
11 checks passed
@armansito armansito deleted the bump-estimate-segments branch March 15, 2024 22:18
@armansito armansito restored the bump-estimate-segments branch March 15, 2024 22:31
@armansito armansito deleted the bump-estimate-segments branch March 16, 2024 21:09
hubot pushed a commit to google/skia that referenced this pull request Apr 2, 2024
The bump allocation estimator computes a conservative estimate of the
required buffer allocations based on heuristics over the encoded path
data. See the description of vello PR #454 for some examples scenes and
the current factors of overestimation: linebender/vello#454

Bug: b/285193099
Change-Id: I131305914307591ad04d6158f24a94235826ac10
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/828997
Commit-Queue: Arman Uguray <armansito@google.com>
Reviewed-by: Jim Van Verth <jvanverth@google.com>
Reviewed-by: James Godfrey-Kittle <jamesgk@google.com>
hubot pushed a commit to google/skia that referenced this pull request Apr 2, 2024
This reverts commit d63b489.

Reason for revert: path_huge_crbug_800804 is causing overflow due to multiply in the estimator code. This requires a fix in a third_party repo so fixing forward will take time.

Original change's description:
> [vello] Enable the bump allocation estimator
>
> The bump allocation estimator computes a conservative estimate of the
> required buffer allocations based on heuristics over the encoded path
> data. See the description of vello PR #454 for some examples scenes and
> the current factors of overestimation: linebender/vello#454
>
> Bug: b/285193099
> Change-Id: I131305914307591ad04d6158f24a94235826ac10
> Reviewed-on: https://skia-review.googlesource.com/c/skia/+/828997
> Commit-Queue: Arman Uguray <armansito@google.com>
> Reviewed-by: Jim Van Verth <jvanverth@google.com>
> Reviewed-by: James Godfrey-Kittle <jamesgk@google.com>

Bug: b/285193099
Change-Id: I3604d62d5b55dd4680ee2a8dc49e37cfdac01500
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/835421
Auto-Submit: Arman Uguray <armansito@google.com>
Commit-Queue: Arman Uguray <armansito@google.com>
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants