Skip to content

bug: format() produces scientific notation for exact powers of 10 #1

@underwoo

Description

@underwoo

Summary

__format__ uses math.ceil(math.log10(n)) to compute the precision passed to Python's g format. For values that are exact powers of 10 (10.0, 100.0, 1000.0, …), log10 returns an exact integer and ceil of that integer equals the g-format exponent. Python's g format switches to scientific notation when exponent >= precision, so these values always render in scientific notation.

The docstring for __format__ explicitly states: "FSize will attempt to not display the number in scientific notation."

Reproduction

from fsize import FSize

format(FSize(10 * 1024), "K")   # '1e+01'  — expected '10'
format(FSize(100 * 1024), "K")  # '1e+02'  — expected '100'
format(FSize(1000 * 1024), "K") # '1e+03'  — expected '1000'

Root cause

src/fsize/__init__.py, line 207:

log_digits = math.ceil(math.log10(n)) if n > 0 else 0

For n = 10.0: ceil(log10(10.0)) = ceil(1.0) = 1. Then format(10.0, ".1g") uses scientific because exponent (1) ≥ precision (1).

Fix

Use math.floor(math.log10(n)) + 1 instead of math.ceil(math.log10(n)). This ensures precision always exceeds the exponent by at least 1:

  • n = 10.0: floor(1.0) + 1 = 2format(10.0, ".2g") = "10"
  • n = 100.0: floor(2.0) + 1 = 3format(100.0, ".3g") = "100"
  • n = 1024.0: floor(3.0103) + 1 = 4format(1024.0, ".4g") = "1024"

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions