Skip to content

Decimal/Floating point number arithmetic #5

@Kolterdyx

Description

@Kolterdyx

This issue is here to discuss decimal number arithmetic (floating/fixed point numbers) implementation.

Current implementation is whacky at best. It uses fixed point numbers and it has trouble with big numbers because it has to give up digits for decimal values. Addition and subtraction work fine, but multiplication and division cause a lot of trouble since either one or both of the numbers in the operation have to be scaled up or down by 10 to the power of however many digits of precision we have, which causes even more overflow issues. The only good thing is that it's almost as efficient as integer arithmetic (because it technically IS integer arithmetic). The implementation was severely limited by the fact that commands can't return floating point values, which means I couldn't store actual float/double values in storage, and then scale them up or down when needed. This is sadly working as intended so I doubt I'll be able to fix this without a huge overhaul implementing my very own data type.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requesthelp wantedExtra attention is needed

    Projects

    Status

    Done

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions