Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

lanczos3: Const-fold offset calculation to hardcoded 0.5 #28

Merged
merged 1 commit into from
Aug 23, 2023

Commits on Aug 23, 2023

  1. lanczos3: Const-fold offset calculation to hardcoded 0.5

    The usefulness of this magic took a fair bit of time to understand,
    while we can trivially remove it after deducing that it always computes
    to the constant `0.5`, and gets rid of some strange bright spots in the
    center of our image compared to #26.
    
    Before:
    
    ![square_test_result](https://github.com/Traverse-Research/ispc-downsampler/assets/2325264/273556b4-6f53-43d5-9424-31fef5ca7966)
    
    After:
    
    ![square_test_result](https://github.com/Traverse-Research/ispc-downsampler/assets/2325264/e0507eee-9a58-4fd8-b9fc-7a0a3c485ee8)
    
    First, we start by knowing that `uv` is divided by `target_size` before
    it is passed to `resample_internal()`.  Hence, if we multiply it by
    `target_size` again, there should be no fractional part and
    `center_pixel` always becomes `0`.  Floating point rounding errors being
    gone now, this is what solves the bright spots in the center of the
    image mentioned above.
    
    Then we are left with:
    
        center = uv - (0.0 - 0.5) * inv_target_size
    
    Which becomes:
    
        center = uv + 0.5 * inv_target_size
    
    As a drive-by cleanup we can now see that `(inv_)target_size` is only
    used to offset `uv` by another half _target_ pixel to point to the
    center instead of the top-left.  These values were already involved in
    converting the `uv` coordinate from target pixels to normalized
    coordinates, so it reads more logical (involving less math) to factor
    this calculation into the call site and remove two extraneous function
    parameters from `resample_internal()` as a result.
    
    Now, continuing our journey, plug this into `offset` and simplify:
    
        offset = (uv - center) * target_size
        offset = (uv - (uv + 0.5 * inv_target_size)) * target_size
        offset = (-0.5 * inv_target_size) * target_size
        offset = -0.5
    
    And we have our target value.  Then, because they are subtracted when
    calling `lanczos3_filter()`, we turn this into positive `0.5`.
    
    Note that I have _zero_ clue whether this is the right value, but when
    sampling a 6x6 grid (not 7x7 as thought in #27) we only visit pixel
    positions `[-3, ..., 2]`, thus neatly retrieving weights at `[-2.5, ...,
    2.5]` and never hitting the `3.5` value which is above `3` where
    `lanczos3_filter(3.5)` returns `0.`.
    MarijnS95 committed Aug 23, 2023
    Configuration menu
    Copy the full SHA
    5a21d54 View commit details
    Browse the repository at this point in the history