Skip to content

TimeDelta.__repr__ displays excess precision #100

@bkeryan

Description

@bkeryan

Bug Report

nitypes.bintime.TimeDelta.__repr__ currently displays a constructor expression using decimal.Decimal, but it doesn't limit the decimal to an appropriate number of digits.

Repro or Code Sample

>>> import nitypes.bintime as bt
>>> from decimal import Decimal
>>> x = bt.TimeDelta(Decimal("1.23456789"))
>>> x
nitypes.bintime.TimeDelta(Decimal('1.234567889999999999977708248222185716258536558598279953002929688'))
>>> x._ticks
22773757908449605611
>>> y = bt.TimeDelta(Decimal('1.234567889999999999977708248222185716258536558598279953002929688'))
>>> y._ticks
22773757908449605611
>>> z = bt.TimeDelta(Decimal('1.2345678899999999999777'))
>>> z._ticks
22773757908449605611

Expected Behavior

The smallest time increment that NI-BTF can represent is 54210 yoctoseconds, so there should be no more than 24 decimal places (yoctoseconds), and preferably less.

Current Behavior

64 decimal places are displayed, because that is the precision that bintime uses to do decimal conversions.

Possible Solution

Use format strings to specify the decimal precision:

>>> "{0:.024}".format(Decimal('1.234567889999999999977708248222185716258536558598279953002929688'))
'1.23456788999999999997771'

Context

It makes the output of repr harder to read.

Your Environment

  • nitypes version: 0.1.0-dev1
  • Python version: 3.11.9

AB#3159517

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions