Error: Balance assertion off by $-0.00

[maxnikulin]

I have puzzled by the following error message having just zeroes in discrepancy amount and I have no idea how to debug the issue:

While parsing file "/tmp/roundto.ledger", line 5:
While parsing posting:
  Account  0  = $-700.67
                ^^^^^^^^
Error: Balance assertion off by $-0.00 (expected to see $-700.67)

the test file is

2024-02-01 Transaction
        Account   (-roundto($12.34 * 56.78, 2))
        Expenses  $700.67
2024-02-02 Assertion
        Account  0  = $-700.67

The idea is to check that billing is correct: per unit price, volume and total amount are in agreement.

An attempt to specify excessive precision

commodity $
        format $1000.0000000000000

failed with a similar message with more zeroes

While parsing file "/tmp/roundto.ledger", line 5:
While balancing transaction from "/tmp/roundto.ledger", lines 3-5:
> 2024-02-01 Transaction
>       Account   (-roundto($12.34 * 56.78, 2))
>       Expenses  $700.67
Unbalanced remainder is:
    $0.0000000000000
Amount to balance against:
  $700.6700000000000
Error: Transaction does not balance

ledger --version
Ledger 3.3.0-20230208, the command-line accounting tool

Earlier (ledger-3.1.3) I used a bit annoying trick with no roundto call and an extra posting to correct rounding and an automatic transaction adding a virtual posting to cancel rounding correction. Currently another trick is required since virtual and regular accounts have diverged and assertions work in different way (#1959).

How to find the real discrepancy hidden behind zeroes?

[igbanam]

This is a precision problem.

I read somewhere — maybe in the docs — that ledger defaults to 6 places of precision. Stepping through this example, the diff is below.

(lldb) p amt.to_double()
(double) -700.66999999999996
(lldb) p diff.to_amount().to_double()
(double) -4.0927261579781769E-14
(lldb)

ledger/src/textual.cc

Lines 1733 to 1741 in d4e3c6b

	// balance assertion
	diff -= post->amount.strip_annotations(keep_details_t());
	if (! no_assertions && ! diff.is_zero()) {
	balance_t tot = (-diff + amt).strip_annotations(keep_details_t());
	DEBUG("textual.parse", "Balance assertion: off by " << diff << " (expected to see " << tot << ")");
	throw_(parse_error,
	_f("Balance assertion off by %1% (expected to see %2%)")
	% diff.to_string() % tot.to_string());
	}

[maxnikulin]

This is a precision problem.

I read somewhere — maybe in the docs — that ledger defaults to 6 places of precision.

My expectation from user point of view is that roundto() result should match parsing decimal number having similar precision. IIRC converting from decimal numbers is deterministic in IEEE 754. Perhaps it is something with low level rounding mode in roundto() implementation.

Stepping through this example, the diff is below.

Thanks, however my another expectation is that a tool should provide enough information to figure out what happens without running a debugger. I have ledger installed using apt install ledger. Likely I can add the repository with debug symbols, but I suspect it would still inconvenient with configure debug option disabled. I would prefer to avoid building tools from source.

(lldb) p diff.to_amount().to_double()
(double) -4.0927261579781769E-14

ledger/src/textual.cc
% diff.to_string() % tot.to_string());

Reporting diff as zero is not helpful here. -4e-14 would be at least some clue. However it is a pure bug and user can not do anything anyway then 0.00 is acceptable.

As a workaround, I have found a way to correct rounding errors by explicit positngs. A file worked with earlier versions required enough edits and care concerning virtual and non-virtual facets of accounts.

2024-02-01 Transaction
	Account   (-$12.34 * 56.78)
	Account   $-0.0048  ; :discrepancy:
	Expenses  $700.67
2024-02-02 Assertion
	Account  0  = $-700.67

[tbm]

[message deleted. I think I misunderstood the problem. I don't know what's going on here]

[maxnikulin]

(lldb) p amt.to_double()
(double) -700.66999999999996

This is expected value for 700.67. Calling in_place_roundto(2) for 700.66520000000003 gives the same double value. Unfortunately I am not familiar with underlying gmp types to reason if values are correct.

My naive attempts to reproduce the issue in python have failed

from gmpy2 import mpfr
mpfr(math.ceil(float(mpfr(12.34)*mpfr(56.78))*100.-0.49999999)/pow(10., 2)) - mpfr(700.67)
# mpfr('0.0')
from gmpy2 import mpq
mpq(math.ceil(float(mpq(12.34)*mpq(56.78))*100.-0.49999999)/pow(10., 2)) - mpq(700.67)
# mpq(0,1)

I have tried the ledger-dbgsym package, but debugging of optimized build is inconvenient.

Even shorter example causing the trouble:

assert roundto(12.34 * 56.78, 2) - 700.67 == 0

However the following one assertion is passed

assert roundto(12.34 * 56.78, 2) - roundto(700.67, 2) == 0

[maxnikulin]

; pass
assert roundto(1.5, 1) == 1.5
; fail
assert roundto(1.1, 1) == 1.1

[tbm]

Adding @the-solipsist who had good input on #1983

[igbanam]

I looked into the output of roundto($12.34 * 56.78, 2)

(lldb) p result.to_string()
(ledger::string) "$700.67"
(lldb) p result.to_amount().to_double()
(double) 700.66520000000003

And yes, it's something with the underlying library handling the rational number.

[maxnikulin]

(lldb) p result.to_string()
(ledger::string) "$700.67"
(lldb) p result.to_amount().to_double()
(double) 700.66520000000003

@igbanam, I suspect it is not roundto() result.

[maxnikulin]

I have figured out what happens. Minimal example:

ledger -f /dev/null eval '(700.67 - roundto(700.67, 2))*10,000,000,000,000,000'
409.27

Current amount_t::in_place_roundto(int) implementation sets multiple precision value quantity->val from double result. It causes huge power 2 denominator instead of exact value with power 10 one having given numbers of digits:

>>> from gmpy2 import mpq
>>> import math
>>> qs = mpq("700.67"); qs
mpq(70067,100)
>>> qr = mpq(700.67); qr
mpq(6163158497870479,8796093022208)
>>> math.frexp(qr.denominator)
(0.5, 44)
>>> 2**43
8796093022208
>>> float(qs) - float(qr)
0.0
>>> float(qs - qr)
4.092726157978177e-14

I think, something close to gmpy2 implementation without libc functions dealing with double should be used

>>> round(mpq(12.34)*mpq(56.78), 2)
mpq(70067,100)

A couple of related questions:

• Should amount_t::in_place_roundto(int) update quantity->prec?
• Why first transaction in the original reproducer does not cause balance error? It would be better than postponed account assertion failure.

I still believe that reporting discrepancy of 0 is not helpful, actual number should appear even if it is not consistent with current format.

[maxnikulin]

It seems #2361 fixes primary cause of the issue. I can not say that I am confident with the code since it is created using monkey typing approach.

Other questions like updating precision or reporting discrepancy as zero remains.