Skip to content

Pre-calculated encode/decode output sizes return incorrect values due to 64-bit number overflow #182

@05nelsonm

Description

@05nelsonm

Calculated values can experience a number overflow and return 0, in the event the first part of the calculation is multiplication which pushes the 64-bit integer (Long) negative.

println(Long.MAX_VALUE * 6L / 8L)
// 0
println((Long.MAX_VALUE.toDouble() * 6.0 / 8.0).toLong())
// 6917529027641081856

Note that with 32-bit integer (Int), the calculation succeeds if converted to 64-bit integer b/c Int.MAX_VALUE * 6 is less than Long.MAX_VALUE, so will fit.

A failure example is Base64.Default.config.encodeOutSize(Long.MAX_VALUE returning 0 currently. Base32 is also affected.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions