New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

print should preserve implicits amounts the way the user wrote them #442

Closed
peti opened this Issue Dec 19, 2016 · 14 comments

Comments

Projects
None yet
3 participants
@peti
Contributor

peti commented Dec 19, 2016

When I print a journal file with hledger, then the output gives the amount for both sides of the (trivial) transaction like this:

2016/12/19 Jasmin
    Ausgaben:Restaurant        EUR 13.00
    Vermögen:Bargeld:Peter    EUR -13.00

If I print that same journal with the C++ ledger, however, then it omits the amount from the second part of the transaction and prints just:

2016/12/19 Jasmin
    Ausgaben:Restaurant                    EUR 13.00
    Vermögen:Bargeld:Peter

Personally, I feel that the second for is neater because it omits redundant information, so now I wonder whether there's a way to make hledger print output in the same way?

@simonmichael

This comment has been minimized.

Show comment
Hide comment
@simonmichael

simonmichael Dec 20, 2016

Owner

I seem to remember some reason for changing it to the current behaviour - making conversion prices explicit or something ? There's no option to change it at the moment.

Owner

simonmichael commented Dec 20, 2016

I seem to remember some reason for changing it to the current behaviour - making conversion prices explicit or something ? There's no option to change it at the moment.

simonmichael added a commit that referenced this issue Jan 10, 2017

print: omit the last posting amount, as we used to #465, #442
This avoids printing invalid journal format for entries where an implicit amount has multiple commodities.
@ony

This comment has been minimized.

Show comment
Hide comment
@ony

ony Jan 10, 2017

Collaborator

I don't think this is a good behavior. I like ledger because it preserves what user write and allows to get all kinds of reports on it.
This feature results in making transaction written by user with explicit amounts more implicit.
I don't mind if ledger print will preserve original content. I.e. if user omits number it also omits it in output. And I don't mind much if it puts omitted number to make it more explicit. But I definitely don't like when it drops something I wrote.
@peti what this feature gives to user? What is the use-case? If it really required for some work-flow can we make it optional and disabled by default?
I can see next example that is against it as a default behavior:

  1. User have old journal.
2016/12/19 Jasmin
    Ausgaben:Restaurant        EUR 13.00
    Vermögen:Bargeld:Peter    EUR -13.00
  1. User use hledger print to generate clean journal in a proper order and saves it to file.
2016/12/19 Jasmin
    Ausgaben:Restaurant                    EUR 13.00
    Vermögen:Bargeld:Peter
  1. User remembers that he forgot to add another expense and update freshly generated file.
2016/12/19 Jasmin
    Ausgaben:Restaurant                    EUR 13.00
    Ausgaben:Tips                           EUR 1.00
    Vermögen:Bargeld:Peter

With this behavior, all unbalanced values will go to implicit entry which not always correct.

With all numbers specified explicitly, he'll need to update some of the numbers or even add another entry to make transaction balance (i.e. add Vermögen:Bargeld:Jane to balance tips). I.e.

2016/12/19 Jasmin
    Ausgaben:Restaurant                    EUR 13.00
    Ausgaben:Tips                           EUR 1.00
    Vermögen:Bargeld:Peter                EUR -13.00

Will require fixing

2016/12/19 Jasmin
    Ausgaben:Restaurant                    EUR 13.00
    Ausgaben:Tips                           EUR 1.00
    Vermögen:Bargeld:Peter                EUR -13.00
    Vermögen:Bargeld:Jane                  EUR -1.00
Collaborator

ony commented Jan 10, 2017

I don't think this is a good behavior. I like ledger because it preserves what user write and allows to get all kinds of reports on it.
This feature results in making transaction written by user with explicit amounts more implicit.
I don't mind if ledger print will preserve original content. I.e. if user omits number it also omits it in output. And I don't mind much if it puts omitted number to make it more explicit. But I definitely don't like when it drops something I wrote.
@peti what this feature gives to user? What is the use-case? If it really required for some work-flow can we make it optional and disabled by default?
I can see next example that is against it as a default behavior:

  1. User have old journal.
2016/12/19 Jasmin
    Ausgaben:Restaurant        EUR 13.00
    Vermögen:Bargeld:Peter    EUR -13.00
  1. User use hledger print to generate clean journal in a proper order and saves it to file.
2016/12/19 Jasmin
    Ausgaben:Restaurant                    EUR 13.00
    Vermögen:Bargeld:Peter
  1. User remembers that he forgot to add another expense and update freshly generated file.
2016/12/19 Jasmin
    Ausgaben:Restaurant                    EUR 13.00
    Ausgaben:Tips                           EUR 1.00
    Vermögen:Bargeld:Peter

With this behavior, all unbalanced values will go to implicit entry which not always correct.

With all numbers specified explicitly, he'll need to update some of the numbers or even add another entry to make transaction balance (i.e. add Vermögen:Bargeld:Jane to balance tips). I.e.

2016/12/19 Jasmin
    Ausgaben:Restaurant                    EUR 13.00
    Ausgaben:Tips                           EUR 1.00
    Vermögen:Bargeld:Peter                EUR -13.00

Will require fixing

2016/12/19 Jasmin
    Ausgaben:Restaurant                    EUR 13.00
    Ausgaben:Tips                           EUR 1.00
    Vermögen:Bargeld:Peter                EUR -13.00
    Vermögen:Bargeld:Jane                  EUR -1.00
@simonmichael

This comment has been minimized.

Show comment
Hide comment
@simonmichael

simonmichael Jan 10, 2017

Owner

This I agree with: we shouldn't remove explicitness that the user may have put there intentionally, eg for error checking and readability.

Owner

simonmichael commented Jan 10, 2017

This I agree with: we shouldn't remove explicitness that the user may have put there intentionally, eg for error checking and readability.

@peti

This comment has been minimized.

Show comment
Hide comment
@peti

peti Jan 10, 2017

Contributor

There is no one true way for hledger to behave in this case, IMHO. As far as I am concerned, the behavior should be configurable. My personal preference is for hledger to keep the journal the way I wrote it -- i.e. don't insert or remove any amounts. Others may have other preference (i.e. make all posting amounts explicit).

Contributor

peti commented Jan 10, 2017

There is no one true way for hledger to behave in this case, IMHO. As far as I am concerned, the behavior should be configurable. My personal preference is for hledger to keep the journal the way I wrote it -- i.e. don't insert or remove any amounts. Others may have other preference (i.e. make all posting amounts explicit).

@simonmichael

This comment has been minimized.

Show comment
Hide comment
@simonmichael

simonmichael Jan 10, 2017

Owner

Following helpful discussion here and on #465 and #hledger, I think the priorities for print are now:

  1. don't remove explicitness the user has added
  2. print valid journal format by default, adding whatever explicitness needed to do that easily
  3. preserve the user's original implicit/explicit style when it does not conflict with 2
Owner

simonmichael commented Jan 10, 2017

Following helpful discussion here and on #465 and #hledger, I think the priorities for print are now:

  1. don't remove explicitness the user has added
  2. print valid journal format by default, adding whatever explicitness needed to do that easily
  3. preserve the user's original implicit/explicit style when it does not conflict with 2
@simonmichael

This comment has been minimized.

Show comment
Hide comment
@simonmichael

simonmichael Jan 11, 2017

Owner

With #466, 1 and 2 should now be covered, and 3 remains open for PRs and design decisions, like when/whether to activate various modes such as:

  • add sufficient explicitness for correct output (current default)
  • preserve original implicit/explicit entries as far as possible
  • make all entries as implicit as possible
  • make all entries as explicit as possible
Owner

simonmichael commented Jan 11, 2017

With #466, 1 and 2 should now be covered, and 3 remains open for PRs and design decisions, like when/whether to activate various modes such as:

  • add sufficient explicitness for correct output (current default)
  • preserve original implicit/explicit entries as far as possible
  • make all entries as implicit as possible
  • make all entries as explicit as possible
@peti

This comment has been minimized.

Show comment
Hide comment
@peti

peti Jan 11, 2017

Contributor

hledger cannot insert amounts into places where one is required. The posting amount is required because the posting would be ambiguous otherwise (i.e. in the presence of a balance assertion), so by definition hledger cannot figure out the proper amount to insert. Am I missing something?

Contributor

peti commented Jan 11, 2017

hledger cannot insert amounts into places where one is required. The posting amount is required because the posting would be ambiguous otherwise (i.e. in the presence of a balance assertion), so by definition hledger cannot figure out the proper amount to insert. Am I missing something?

@simonmichael

This comment has been minimized.

Show comment
Hide comment
@simonmichael

simonmichael Jan 11, 2017

Owner

I haven't thought about balance assertions & assignments relating to this issue, assuming all that should work normally. Can you give an example of the problem you're seeing ?

Owner

simonmichael commented Jan 11, 2017

I haven't thought about balance assertions & assignments relating to this issue, assuming all that should work normally. Can you give an example of the problem you're seeing ?

@peti

This comment has been minimized.

Show comment
Hide comment
@peti

peti Jan 12, 2017

Contributor

I have a hand-edit ledger file, foo.ledger, from which I generate various other ledger files with automatic processes (like hledger-interest). Now main.ledger includes all of them in the correct order (the generated ones before the manually edited one). Now, foo.ledger contains balance assertions for accounts that contain automatically generated transactions (like interest). Thus, whether those assertions hold or fail depends on whether the other generated files are up-to-date or not. If hledger were to infer posting amounts such that the assertion holds, then that amount can be false.

I actually ran into this problem before and realized that postings with a balance assertion must always give the amount, too, to be stable, so for my particular ledger this problem would not occur, but still ... if that issue can arise in principle, then I'm sure someone has thought up an even weirder process that will trigger it, too.

Another open question about balance assertions would be how to deal with them when running as hledger print -I? That flag means that trying to infer amounts for balance assertions is off the table, right?

Anyway, IMHO hledger should never mess with postings that specify a balance assertion -- -I flag or not.

Contributor

peti commented Jan 12, 2017

I have a hand-edit ledger file, foo.ledger, from which I generate various other ledger files with automatic processes (like hledger-interest). Now main.ledger includes all of them in the correct order (the generated ones before the manually edited one). Now, foo.ledger contains balance assertions for accounts that contain automatically generated transactions (like interest). Thus, whether those assertions hold or fail depends on whether the other generated files are up-to-date or not. If hledger were to infer posting amounts such that the assertion holds, then that amount can be false.

I actually ran into this problem before and realized that postings with a balance assertion must always give the amount, too, to be stable, so for my particular ledger this problem would not occur, but still ... if that issue can arise in principle, then I'm sure someone has thought up an even weirder process that will trigger it, too.

Another open question about balance assertions would be how to deal with them when running as hledger print -I? That flag means that trying to infer amounts for balance assertions is off the table, right?

Anyway, IMHO hledger should never mess with postings that specify a balance assertion -- -I flag or not.

@ony

This comment has been minimized.

Show comment
Hide comment
@ony

ony Jan 12, 2017

Collaborator

@peti I can see now what you mean. When I tried to use multiple files with assignment/assertion I've hit a problem with intra-day ordering. But looks like all your assertions/assignments are the last postings of the day.

While print doesn't preserve user implicitness you still can solve this problem.
Just keep all of your assignments in a separate hand-edited file and avoid post-processing it.
It should be ok if your get pipeline like ((foo.ledger -> post-process) + foo-assignments.ledger) -> ledger-interest -> ledge-rewrite for budgeting.

I agree that print should preserve original implicitness by default. If you don't mind I'd like to treat this issue ticket exactly for that. And not like forcing every transaction to have one posting with omitted amount.
Can we change subject to reflect that?

P.S. Note that C++ ledger preserves original user input.

Collaborator

ony commented Jan 12, 2017

@peti I can see now what you mean. When I tried to use multiple files with assignment/assertion I've hit a problem with intra-day ordering. But looks like all your assertions/assignments are the last postings of the day.

While print doesn't preserve user implicitness you still can solve this problem.
Just keep all of your assignments in a separate hand-edited file and avoid post-processing it.
It should be ok if your get pipeline like ((foo.ledger -> post-process) + foo-assignments.ledger) -> ledger-interest -> ledge-rewrite for budgeting.

I agree that print should preserve original implicitness by default. If you don't mind I'd like to treat this issue ticket exactly for that. And not like forcing every transaction to have one posting with omitted amount.
Can we change subject to reflect that?

P.S. Note that C++ ledger preserves original user input.

@peti peti changed the title from Omit redundant amounts from printed transactions? to print should preserve implicits amounts the way they were written by the user Jan 12, 2017

@peti peti changed the title from print should preserve implicits amounts the way they were written by the user to print should preserve implicits amounts the way the user wrote them Jan 12, 2017

@peti

This comment has been minimized.

Show comment
Hide comment
@peti

peti Jan 12, 2017

Contributor

Just keep all of your assignments in a separate hand-edited file and avoid post-processing it.

I'm not sure whether I follow. What do you mean by "assignments"?

I agree that print should preserve original implicitness by default. Can we change subject to reflect that?

I edited the title to reflect that goal.

Contributor

peti commented Jan 12, 2017

Just keep all of your assignments in a separate hand-edited file and avoid post-processing it.

I'm not sure whether I follow. What do you mean by "assignments"?

I agree that print should preserve original implicitness by default. Can we change subject to reflect that?

I edited the title to reflect that goal.

@ony

This comment has been minimized.

Show comment
Hide comment
@ony

ony Jan 12, 2017

Collaborator

@peti I use next terminology

2017/1/1
    Assets:Cash:Wallet  = $20  ; balance assignment: implicit delta with explicit balance
    Equity:Unbalanced  ; implicit amount: implicit delta with implicit/absent balance
    Assets:Bank  0 = $1,000.00  ; balance assertion: explicit delta with explicit balance
    Liabilities  $-5  ; general posting: explicit delta with implicit/absent balance
  • Assets:Cash:Wallet - gets implicit amount calculated from balance difference
  • Equity:Unbalanced - gets implicit amount calculated from whole transaction balance (to make it balance to zero)
  • Equity:Unbalanced -- Assets:Bank - gets assertion check that can be turned off by -I

Thank you for updating subject.

Collaborator

ony commented Jan 12, 2017

@peti I use next terminology

2017/1/1
    Assets:Cash:Wallet  = $20  ; balance assignment: implicit delta with explicit balance
    Equity:Unbalanced  ; implicit amount: implicit delta with implicit/absent balance
    Assets:Bank  0 = $1,000.00  ; balance assertion: explicit delta with explicit balance
    Liabilities  $-5  ; general posting: explicit delta with implicit/absent balance
  • Assets:Cash:Wallet - gets implicit amount calculated from balance difference
  • Equity:Unbalanced - gets implicit amount calculated from whole transaction balance (to make it balance to zero)
  • Equity:Unbalanced -- Assets:Bank - gets assertion check that can be turned off by -I

Thank you for updating subject.

ony added a commit to ony/hledger that referenced this issue Jan 12, 2017

@simonmichael

This comment has been minimized.

Show comment
Hide comment
@simonmichael

simonmichael Jan 12, 2017

Owner

@peti thanks for the example. I don't see it yet, can you show a reproducible one ?

-I/--ignore-assertions simply means "don't raise an error if you find a false balance assertion", I don't think it changes any other behaviour.

Owner

simonmichael commented Jan 12, 2017

@peti thanks for the example. I don't see it yet, can you show a reproducible one ?

-I/--ignore-assertions simply means "don't raise an error if you find a false balance assertion", I don't think it changes any other behaviour.

ony added a commit to ony/hledger that referenced this issue Jan 12, 2017

Use original postings for hledger print
- Remember original postings in inferFromAssignment,
  inferBalancingAmount, inferBalancingPrices.
- User original posting in matchesPosting only for tags and account.
  Otherwise we wouldn't be able to query on amt:<0 for postings with
  implicit amount.
- Introduce "--explicit" option for "print" command which brings back
  old behavior when every inferred number being printed.
- Make "print" by default print original postings without inferred
  amounts. But use effective account name to have effect from aliases.
- Instruct shell tests with an new expected output or to use
  --explicit option when inferred amounts are checked.

Resolves simonmichael#442

ony added a commit to ony/hledger that referenced this issue Jan 12, 2017

Use original postings for hledger print
- Remember original postings in inferFromAssignment,
  inferBalancingAmount, inferBalancingPrices.
- User original posting in matchesPosting only for tags and account.
  Otherwise we wouldn't be able to query on amt:<0 for postings with
  implicit amount.
- Introduce "--explicit" option for "print" command which brings back
  old behavior when every inferred number being printed.
- Make "print" by default print original postings without inferred
  amounts. But use effective account name to have effect from aliases.
- Instruct shell tests with an new expected output or to use
  --explicit option when inferred amounts are checked.

Resolves simonmichael#442
@ony

This comment has been minimized.

Show comment
Hide comment
@ony

ony Jan 12, 2017

Collaborator

@peti, you can try branch feature/442 on my fork. Change in behavior mostly exposed in tests tests/print/explicit.test
@simonmichael, I'll try to rebase this off from --pivot desc feature (#460).

Collaborator

ony commented Jan 12, 2017

@peti, you can try branch feature/442 on my fork. Change in behavior mostly exposed in tests tests/print/explicit.test
@simonmichael, I'll try to rebase this off from --pivot desc feature (#460).

ony added a commit to ony/hledger that referenced this issue Jan 12, 2017

Use original postings for hledger print
- Introduce "--explicit" option for "print" command which brings back
  old behavior when every inferred number being printed.
- Make "print" by default print original postings without inferred
  amounts. But use effective account name to have effect from aliases.
- Instruct shell tests with an new expected output or to use
  --explicit option when inferred amounts are checked.

Resolves simonmichael#442

ony added a commit to ony/hledger that referenced this issue Jan 12, 2017

Use original postings for hledger print
- Introduce "--explicit" option for "print" command which brings back
  old behavior when every inferred number being printed.
- Make "print" by default print original postings without inferred
  amounts. But use effective account name to have effect from aliases.
- Instruct shell tests with an new expected output or to use
  --explicit option when inferred amounts are checked.

Resolves simonmichael#442

ony added a commit to ony/hledger that referenced this issue Jan 13, 2017

Use original postings for hledger print
- Introduce "--explicit" option for "print" command which brings back
  old behavior when every inferred number being printed.
- Make "print" by default print original postings without inferred
  amounts. But use effective account name to have effect from aliases.
- Instruct shell tests with an new expected output or to use
  --explicit option when inferred amounts are checked.

Resolves simonmichael#442

simonmichael added a commit that referenced this issue Jan 13, 2017

Use original postings for hledger print
- Introduce "--explicit" option for "print" command which brings back
  old behavior when every inferred number being printed.
- Make "print" by default print original postings without inferred
  amounts. But use effective account name to have effect from aliases.
- Instruct shell tests with an new expected output or to use
  --explicit option when inferred amounts are checked.

Resolves #442

mstksg added a commit to mstksg/hledger that referenced this issue Feb 3, 2017

print: omit the last posting amount, as we used to simonmichael#465, s…
…imonmichael#442

This avoids printing invalid journal format for entries where an implicit amount has multiple commodities.

mstksg added a commit to mstksg/hledger that referenced this issue Feb 3, 2017

mstksg added a commit to mstksg/hledger that referenced this issue Feb 3, 2017

Preserve implicit amounts and prices the way user wrote them in outpu…
…t of print command (simonmichael#471)

* Remember original postings during infer and pivot

This includes such functions like:
- inferFromAssignment
- inferBalancingAmount
- inferBalancingPrices
- pivotPosting

* Use original postings for hledger print

- Introduce "--explicit" option for "print" command which brings back
  old behavior when every inferred number being printed.
- Make "print" by default print original postings without inferred
  amounts. But use effective account name to have effect from aliases.
- Instruct shell tests with an new expected output or to use
  --explicit option when inferred amounts are checked.

Resolves simonmichael#442

mstksg added a commit to mstksg/hledger that referenced this issue Feb 3, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment