Hledger does not handle balance assertions when multiple commodities #195

Closed
thdox opened this Issue Jun 22, 2014 · 13 comments

Projects

None yet

2 participants

@thdox
thdox commented Jun 22, 2014

Please see below:

  • the test file
  • works correctly with ledger (c++)
  • fails with hledger

$ cat hledger_test_16

2000/01/01 * Initial
    Actif:Courant:BnpCc                 -102411,12 F = -102411,12 F
    Actif:Courant:BnpCc                   21036,81 € = 21036,81 €
    Equity

2002/01/11 * Passage à l'euro
    Actif:Courant:BnpCc                  102411,12 F = 0,00 F
    Actif:Courant:BnpCc                  -15612,47 € = 5424,34 €
    Revenu:Devise                       -102411,12 F
    Revenu:Devise                         15612,47 €

$ ledger --no-pager -f hledger_test_16 bal

           5424,34 €  Actif:Courant:BnpCc
         102411,12 F
         -21036,81 €  Equity
        -102411,12 F
          15612,47 €  Revenu:Devise
--------------------
                   0

$ hledger -f hledger_test_16 bal

hledger: Balance assertion failed for account Actif:Courant:BnpCc on 2000-01-01
In transaction:
2000/01/01 * Initial
    Actif:Courant:BnpCc  -102411,12 F
    Actif:Courant:BnpCc    21036,81 €
                          102411,12 F
    Equity                -21036,81 €

After posting:
   Actif:Courant:BnpCc      21036,81 €  ;

expected balance is 21036,81 €, actual balance was -102411,12 F
  21036,81 €.
@simonmichael
Owner

Thanks for the report. I think it's not a bug, as http://hledger.org/manual#balance-assertions explains that multi-commodity balance assertions aren't supported. Ledger's behaviour is interesting - I think it applies each assertion to only one commodity within the overall account balance. Simple enough when you understand it, but perhaps slightly less unintuitive ? How useful is it ?

@thdox
thdox commented Jun 22, 2014
  1. Agreed this is not a bug as documentation explains that multi-commodity balance assertions aren't supported.
  2. BUT, I really wonder why supporting balance assertions if hledger does not support assertions per commodity. Hledger, with this way of working, forces accounts to have a single commodity. Thus restricting the usage of hledger. This breaks bullet 4 of ledger principles: "Ledger is 100% currency-agnostic. You can store multiple currencies in any account, convert between them, or even pay in one currency and receive change in another." Here we are forced to use only one currency, that means hledger knows what is a currency, and thus is not currency agnostic. It knows what it is, it restricts you on this subject.

There are at least two use cases, that I use very frequently in my ledger file:

  1. In Europe, I mean actually euro zone, we had during several years, two official currencies. As an example, I am living in France, and for some period of time the FRF (also noted as 'F' or 'French Franc') and EUR (also noted as '€' or 'Euro') where two currencies that were allowed to be used. This example is described in file from initial description.
  2. Another everyday use case is buying some stocks in broker account

$ cat hledger_test_16a

2011-01-01 * Opening balance
    Assets:Broker                           250.00 GBP = 250.00 GBP
    Equity:Opening balance                 -250.00 GBP

2011-02-01 * Buy 1 AAA for 10.00 GBP
    Assets:Broker                                1 AAA = 1 AAA
    Assets:Broker                           -10.00 GBP

2011-03-01 * Buy 1 AAA for 10.00 GBP
    Assets:Broker                                1 AAA = 2 AAA
    Assets:Broker                           -10.00 GBP

2011-04-01 * Buy 1 BBB for 15.00 GBP
    Assets:Broker                                1 BBB = 1 BBB
    Assets:Broker                           -15.00 GBP

$ hledger -f hledger_test_16a bal

using conversion rules file /home/thierry/VirtualBoxShare/hledger_test_16a.rules
hledger: Balance assertion failed for account Assets:Broker on 2011-02-01
In transaction:
2011/02/01 * Buy 1 AAA for 10.00 GBP
    Assets:Broker  1 AAA @@ 10 GBP
    Assets:Broker    -10.00 GBP

After posting:
   Assets:Broker          1 AAA @@ 10 GBP  ;

expected balance is 1 AAA, actual balance was 1 AAA @@ 10 GBP
     250.00 GBP.

The funny thing here is that error message is "expected balance is 1 AAA", that is exactly what I put in ledger file. But hledger, for whatever reason modify my file by transforming into "1 AAA @@ 10 GBP". This, from my point of view, breaks bullet 1 of ledger principles: "Ledger never creates or modifies your data. Your entries are kept in a text file that you maintain, and you can rest assured, no automated tool will ever change that data.". Well, file is not modified by itself, but what is read is modified. At least hledger reports a transaction that does not exist in the input file (see from line 3 to 6 in error message above).

@thdox
thdox commented Jun 22, 2014

Even if I try to be gentle with hledger, putting Cash aside, hledger still complains, of having two commodities in the same account. This is useful when I regularly buy every months some stock, and I have a running cumulated number of stock shares that I want to track:

$ cat hledger_test_16c

2011-01-01 * Opening balance
    Assets:Broker:Cash                      250.00 GBP = 250.00 GBP
    Equity:Opening balance                 -250.00 GBP

2011-02-01 * Buy 1 AAA for 10.00 GBP
    Assets:Broker                                1 AAA = 1 AAA
    Assets:Broker:Cash                      -10.00 GBP

2011-03-01 * Buy 1 AAA for 10.00 GBP
    Assets:Broker                                1 AAA = 2 AAA
    Assets:Broker:Cash                      -10.00 GBP

2011-04-01 * Buy 1 BBB for 15.00 GBP
    Assets:Broker                                1 BBB = 1 BBB
    Assets:Broker:Cash                      -15.00 GBP

$ hledger -f hledger_test_16c bal

creating default conversion rules file /home/thierry/VirtualBoxShare/hledger_test_16c.rules, edit this file for better results
hledger: Balance assertion failed for account Assets:Broker on 2011-04-01
In transaction:
2011/04/01 * Buy 1 BBB for 15.00 GBP
    Assets:Broker       1 BBB @@ 15 GBP
    Assets:Broker:Cash    -15.00 GBP

After posting:
   Assets:Broker          1 BBB @@ 15 GBP  ;

expected balance is 1 BBB, actual balance was 2 AAA @@ 10 GBP
1 BBB @@ 15 GBP.
@simonmichael
Owner

Hi all,

In current hledger, a balance assertion like

some:account  $1 = $4

asserts that after the $1 posting, some:account's balance is $4.

I learned from #195
that in Ledger, it means that some:account's dollar balance is $4,
and says nothing about any other commodities in the account. This
should be clarified at http://ledger-cli.org/3.0/doc/ledger3.html#Balance-assertions.

I find this a little harder to understand and explain, and unsatisfying
that a balance assertion doesn't definitively nail down
(programmatically and visually) what's in an account. You can add more
postings, with 0 amount if necessary, to assert the balance of each
individual commodity:

some:account  $1 = $4
some:account   0 = EUR 10
some:account   0 = FRF 15

but you're still not absolutely sure of the account's balance - new
commodities can show up there and the assertions won't notice.

Also, what does it mean to assert a commodity-less zero balance ?
Ledger accepts all the assertions below, which seems wrong:

1/1
a 1 = 1
a $1 = $1
a $-1 = 0
a 1 = 0
a 1 = 0
b

The advantage of Ledger's current way is that you can make at least
some assertions about a multi-commodity account. In current hledger
this is explicitly not supported, so if you really needed to assert a
multi-commodity balance you'd have to separate the commodities into
subaccounts.

Compatible balance assertions are important for interoperability
between ledger-likes, so I'm wondering how to harmonise this. I can
make hledger's assertions work like current Ledger, or we could agree
to target a new design. Eg I propose we add a one-line syntax for
multi-commodity amounts, such as:

some:account  $1 = $4, EUR 10, FRF15    ; comma followed by a space separates amounts

and agree that assertions disallow commodities not explicitly
mentioned. What do you think ?

There's at least one other incompatibility between Ledger and hledger
balance assertions:

1/1
  a  1 = 1
  a  1 = 1  ; Ledger expects a to be 1 again, ignoring the previous posting; hledger expects 2
  b

Can I persuade you that hledger's interpretation is better here ?

Either way, I'd also like to have --no-balance-assertions or similar
widely supported, so that you can at least ignore them temporarily
while running another ledger-like on your data.

Thoughts ?

@simonmichael
Owner

Martin>
On Monday, June 23, 2014 6:42:19 PM UTC+2, Martin Blais wrote:

Just to be clear, it also makes sense to do this in Beancount:

2014-06-20 balance Assets:Some:Account 413.43 USD
2014-06-20 balance Assets:Some:Account 201.24 CAD

I too have been wondering if a check on the totality of an inventory might
be useful.
Something like this:

2014-06-20 balance* Assets:Some:Account 413.43 USD, 201.24 CAD

whereby if there are any other commodities in this account at that time
(e.g., EUR) this would raise an error.

Thierry>

Semantically speaking, I would argue that this is two different things:

  • balance assertions, are from my point of view, to assert that balance is
    correct, I mean the amount
  • checking that EUR is not used, is from my point of view a different
    thing, and would be better achieved with a directive like "account
    Assets:Some:Account commodity USD" and "account Assets:Some:Account
    commodity CAD", those two statements would tell that no other
    commodity/currency is allowed in that account.

Thierry

Martin>

That's a good point. This is already in Beancount BTW, as part of the
"open" directive, you can optionally add a list of commodities to restrict
to, e.g.

2010-01-01 open Assets:Some:Account USD,CAD

(The "USD,CAD" bit is optional.)
With this restriction, any transaction posting a change to this account in
a commodity that is not USD nor CAD will trigger an error.

@simonmichael simonmichael added a commit that closed this issue Jul 2, 2014
@simonmichael assert only a single commodity, like Ledger (fixes #195)
This change means you can make assertions on a multi-commodity account
balance (asserting one commodity at a time). On the flip side, you can
no longer assert the complete balance of an account (new unexpected
commodities will not be detected.) We might restore that ability later,
using the == syntax.
8ae303f
@simonmichael
Owner

I've made hledger's assertions work like Ledger's (asserting only a single commodity at a time). The ability to assert the full multi-commodity balance can be added again using the == syntax, if needed.

@thdox
thdox commented Jan 10, 2015

Hi Simon, I've tested this against hledger 0.24.

It works correctly with test cases hledger_test_16a and hledger_test_16c (with an 'a' and a 'c' at the end of filename).
It does not work for hledger_test_16 (without letter at the end of filename).

I suspect this does not work because of your last comment "The ability to assert the full multi-commodity balance can be added again using the == syntax, if needed."
I do not fully catch this last comment. I went to http://hledger.org/manual.html, and a search can not find any occurrence of '=='.

I've tried syntax below, but this fails.

2002/01/11 * Passage à l'euro
    Actif:Courant:BnpCc                  102411,12 F = 0,00 F
    Actif:Courant:BnpCc                  -15612,47 € == 5424,34 €
    Revenu:Devise                       -102411,12 F
    Revenu:Devise                         15612,47 €

Can you help me here by providing a syntax example that would fit with hledger_test_16 test case ?

Thanks

@thdox
thdox commented Jan 10, 2015

Hi Simon,

There is one use case where I always use two balance assertions.
My broker, enable the possibility to sell an ETF, and buy a new one, instantly, without going through cash.
I note this, using ledger c++ syntax, as:

2014/11/12 * Invest
    Actif:Invest:Broker    644,350 ETFone @@ 2000 €
    Actif:Invest   -2000 €

2014/11/12 * Exchange
    Actif:Invest:Broker    -644,350 ETFone @@ 2197,23 €  = 0,000 ETFone  ; @ 3,4100 €
    Actif:Invest:Broker    30,587 ETFtwo @@ 2197,23 €  = 30,587 ETFtwo  ; @ 71,8340 €

Surprisingly, this works, I mean I can use two postings in one transaction, with two balance assertions

Thanks
Thierry

@simonmichael
Owner

Hi Thierry,

== has not been implemented - I meant to suggest we could use that syntax if we want to provide the old (multi-commodity) assertions again.

Good catch with hledger_test_16 - something funny is going on there >:(. Here's a minimal example, commenting either the second posting or second transaction makes it work again:

1/1
    (a)   1F = 1F
    (a)   1G = 1G

1/2
    (a)   1F = 2F
@simonmichael
Owner

Also thanks for the real-world examples, they are very useful. What's an ETF ?

You can have as many assertions as you have postings in a transaction.

I think you know this, but I'll just confirm these ways of writing your exchange transaction all work:

2014/11/12 * Exchange
    Actif:Invest:Broker    -644,350 ETFone @@ 2197,23 €  =  0,000 ETFone
    Actif:Invest:Broker      30,587 ETFtwo @@ 2197,23 €  = 30,587 ETFtwo

or

2014/11/12 * Exchange
    Actif:Invest:Broker    -644,350 ETFone @  3,4100 €  =  0,000 ETFone
    Actif:Invest:Broker      30,587 ETFtwo @ 71,8340 €  = 30,587 ETFtwo

or

2014/11/12 * Exchange
    Actif:Invest:Broker    -644,350 ETFone  =  0,000 ETFone
    Actif:Invest:Broker      30,587 ETFtwo  = 30,587 ETFtwo
@simonmichael simonmichael reopened this Jan 10, 2015
@thdox
thdox commented Jan 10, 2015

Here is link to wikipedia : basically, this is an investment fund, that holds stocks, with low fees.

I was surprised that transaction with ETF was working, as there are two balance assertions, and in hledger_test_16 file, there are also two balance assertions, but this fails. I can see that you re-open this issue #195, so I understand that you accept there is still something to correct. Thanks.

Thierry

@simonmichael
Owner

Yes I will look at it. Thanks for the report.

@simonmichael simonmichael added a commit that referenced this issue Jan 11, 2015
@simonmichael fix balance accumulation across assertions (#195)
A sequence of balance assertions asserting first one commodity, then
another, then the first again, was not working.
257ce57
@simonmichael simonmichael added a commit that referenced this issue Mar 15, 2015
@simonmichael fix balance accumulation across assertions (#195)
A sequence of balance assertions asserting first one commodity, then
another, then the first again, was not working.
6a4f288
@simonmichael
Owner

Fixed by the 0.24.1 release.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment