Follow contract of floor/ceil

[Seelengrab]

floor and ceil are supposed to return the same type as the input type:

floor(x) returns the nearest integral value of the same type as x that is less than or equal to x.

However, floor(N0f8(0.33), digits=1) returns 0.3f0 and ceil(N0f8(0.33), digits=1) returns 0.4f0 i.e. they return Float32.

I haven't tested other types, but since all calls to floor/ceil fallback to the Base definition of floor(x; kwargs...) = round(x, RoundDown; kwargs...) (RoundUp for ceil) and their callchains call float(x), I suspect this is the case for all Normed types.

Found here.

[kimikage]

They are more aptly described as not supporting kwargs at present, rather than the wrong return type.

In order to prioritize this issue, let me clarify on the use cases. Do you want to quantize colors into 10 + 1 (or 10^n + 1) steps? Or in general, do you want to share your code with AbstractFloat?

If it's the former, there will be a more efficient way of not using ceil/floor, and if it's the latter, we will need to support the base option as well. In other words, in the former case, I'm willing to throw errors for the time being.

[Seelengrab]

I'm not the original author of that discourse topic, so I can't really comment on the quantization part, sorry 😅 Just wanted to leave a paper trail :) I've poked the original author though. If they don't mind for now, I don't think you have to prioritize this.

[montanonic]

@kimikage in my usage it was just for quantization reasons, 10+1 as you said, and I can see how there's better ways to do that than ceil/floor, I agree (the problem just came from the first homework assignment for this MIT Julia course https://computationalthinking.mit.edu/Fall20/hw1/).

I think having an error thrown sounds like a good default unless it would cause other issues.

[kimikage]

Exercise! 😄

function _round_digits(x::N0f8, r::RoundingMode, d::Int)
    t(::RoundingMode{:Nearest}) = 0x0081
    t(::RoundingMode{:ToZero})  = 0x0003
    t(::RoundingMode{:Up})      = 0x00ff
    t(::RoundingMode{:Down})    = 0x0003
    xd = x.i * (d === 1 ? 0x000a : 0x0064)
    if d === 1
        x10 = xd + (x.i >> 0x5) + t(r)
        y = (x10 >> 0x8) % UInt8
        return N0f8(y * 0x19 + (y + 0x3) >> 0x2 + y >> 0x2, 0)
    elseif d === 2
        x100 = xd + ((xd + x.i) >> 0x8) + t(r)
        y = (x100 >> 0x8) % UInt8
        z = ((y * 0x233 + 0x212) >> 0xa) % UInt8
        return N0f8(z + y + y - (y === 0x1e) - (y === 0x46), 0)
    else
        return x
    end
end

I'll only support the non-negative digits option. (All other options will throw an error.) I'm going to apply the above optimizations only to N0f8, and use floating point operations for the rest of the Normed. Due to vectorization problems, N0f16 cannot be optimized well. Also, Fixed is still to be tested.

[kimikage]

The difficulty with this is that the digits(or d above) must be on the outermost layer of the broadcast to be a compile-time constant. This means that it needs to be inlined along with the "dying" (not dead) branches. For example, even if d === 1, the cost of the branch for d === 2 would affect the vectorization. 😕
I tried using the Val type, but Julia's specialization of methods with kwargs seems to be incompatible with dynamic Val instances.