New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

transaction modifiers miss inferred amounts in transaction #893

Closed
jkr opened this Issue Oct 9, 2018 · 9 comments

Comments

Projects
None yet
2 participants
@jkr
Contributor

jkr commented Oct 9, 2018

Running with a stack build of up-to-date master.

Transaction modifiers seem to interact oddly with inferred amounts. The following works as expected:

= ^expenses:groceries
    [budget:groceries]                           *-1
    [assets:bank:checking]                        *1

10/7 * MARKET
    expenses:groceries:food                      $20
    assets:bank:checking
$ hledger -ffoo.ledger reg --auto
2018/10/07 MARKET               ex:groceries:food              $20           $20
                                [budget:groceries]            $-20             0
                                [as:bank:checking]             $20           $20
                                assets:bank:checking          $-20             0

But the following does not:

= ^expenses:groceries
    [budget:groceries]                           *-1
    [assets:bank:checking]                        *1

10/7 * MARKET
    expenses:groceries:food
    assets:bank:checking                         $-20
$ hledger -ffoo.ledger reg --auto
hledger: could not balance this transaction - can't have more than one balanced virtual posting with no amount (remember to put 2 or more spaces before amounts)
2018/10/07 * MARKET
    expenses:groceries:food
    [budget:groceries]
    [assets:bank:checking]
    assets:bank:checking                $-20

@simonmichael simonmichael changed the title from transaction modifiers miss some accounts in transaction to transaction modifiers miss inferred amounts in transaction Oct 9, 2018

@simonmichael

This comment has been minimized.

Owner

simonmichael commented Oct 9, 2018

Thanks for the report.

@jkr

This comment has been minimized.

Contributor

jkr commented Oct 9, 2018

I'll poke around as a way of familiarizing myself with the codebase, and see if I can help out. Would HLedger.Data.TransactionModifer be where I should start? (It looks like the inference logic is in H.D.Transaction, right?)

@simonmichael

This comment has been minimized.

Owner

simonmichael commented Oct 9, 2018

@jkr, that would be great. Yes that sounds right.

@jkr

This comment has been minimized.

Contributor

jkr commented Oct 10, 2018

I figured out the issue: it was applying the modifiers before it finalized the journal with journalFinalise. A simple solution (apply modifiers after journalFinalise) is here:

[EDIT: this solution doesn't work, for the reasons discussed in this post. There is a better solution in the post below.]

master...jkr:modifiers_fix

Now this works:

= ^expenses:groceries
    [budget:groceries]                           *-1
    [assets:bank:checking]                        *1

10/7 * MARKET
    expenses:groceries:food                      $20
    assets:bank:checking

10/8 * MARKET2
    expenses:groceries:food
    assets:bank:checking                        -$20
$ stack exec hledger -- print -f /tmp/foo.ledger --auto
2018/10/07 * MARKET
    expenses:groceries:food             $20
    [budget:groceries]                 $-20
    [assets:bank:checking]              $40
    assets:bank:checking               $-20

$ stack exec hledger -- print -f /tmp/foo.ledger --auto
2018/10/07 * MARKET
    expenses:groceries:food             $20
    [budget:groceries]                 $-20
    [assets:bank:checking]              $20
    assets:bank:checking

2018/10/08 * MARKET2
    expenses:groceries:food
    [budget:groceries]                 $-20
    [assets:bank:checking]              $20
    assets:bank:checking               $-20

BUT, this produces its own problem. Because the application happens after the finalization, it no longer checks to make sure they balance. So we can now have:

= ^expenses:groceries
    [budget:groceries]                           *-1
    [assets:bank:checking]                        *2

10/7 * MARKET
    expenses:groceries:food                      $20
    assets:bank:checking                        -$20
$ stack exec hledger -- print -f /tmp/foo.ledger --auto
2018/10/07 * MARKET
    expenses:groceries:food             $20
    [budget:groceries]                 $-20
    [assets:bank:checking]              $40
    assets:bank:checking               $-20

So it seems like the best option would be to run finalize twice: before and after modification. This makes the most intuitive sense since the journal should work both with and without --auto, so it's essentially checking two different journals. I'm not sure how expensive this would be, but the second run would only occur if auto_ iopts, so if you don't run specifiy auto you won't see it.

I could also try to split out infernce from finalize, but I imagine there are other checks that would end up missing, so running it twice if you have auto seems like the most direct route.

Thoughts?

Also: when do you use doctests and when do you use easytests? I'll add tests when i make a PR but I'm not sure what you usually do.

@jkr

This comment has been minimized.

Contributor

jkr commented Oct 10, 2018

Okay, this implements the double-run I discussed above:

master...jkr:modifiers_fix_2

What do you think?

@simonmichael

This comment has been minimized.

Owner

simonmichael commented Oct 10, 2018

That sounds reasonable.

@jkr

This comment has been minimized.

Contributor

jkr commented Oct 10, 2018

OK -- I put up a PR. There's no testing in at the moment, since I couldn't figure out the proper place to put this test in the current framework. If you have a suggestion, I'll add it to the PR.

@jkr

This comment has been minimized.

Contributor

jkr commented Oct 10, 2018

I see from the Travis output where the relevant tests are. I'll try to address those and add to the PR.

jkr added a commit to jkr/hledger that referenced this issue Oct 11, 2018

read: Integrate transaction modifiers with journal finalization
Currently, automated transactions are added before the journal is
finalized. This means that no inferred values will be picked up. We
change the procedure, if `auto_` is set, to

 1. first run `journalFinalise` without assertion checking (assertions
    might be wrong until automated transactions), but with reordering
 2. Insert transaction modifiers
 3. Run `journalFinalise` again, this time with assertion checking as
    set in the options, and without reordering.

If `auto_` is not set, all works as before.

Closes: simonmichael#893
@simonmichael

This comment has been minimized.

Owner

simonmichael commented Oct 12, 2018

Seems good to me! Thanks for the fix.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment