New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

generate automated transactions (postings), like ledger #99

Open
simonmichael opened this Issue Apr 8, 2013 · 13 comments

Comments

Projects
None yet
4 participants
@simonmichael
Owner

simonmichael commented Apr 8, 2013

Original author: simon@joyful.com (March 04, 2013 21:06:26)

We'd like to have automated transactions or postings supported in hledger, eg for automatically generating interest postings based on another posting's amount. We have ledger's design (http://ledger-cli.org/3.0/doc/ledger3.html#Automated-Transactions) but other ideas could be considered.

Original issue: http://code.google.com/p/hledger/issues/detail?id=99


Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

@simonmichael simonmichael added the journal label May 5, 2014

@simonmichael

This comment has been minimized.

Show comment
Hide comment
@simonmichael

simonmichael Nov 15, 2015

Owner

I question this. We have hledger-interest which generates explicit transactions, which is more manual but allows much more control and predictability. Needs discussion/developers, closing.

Owner

simonmichael commented Nov 15, 2015

I question this. We have hledger-interest which generates explicit transactions, which is more manual but allows much more control and predictability. Needs discussion/developers, closing.

@singpolyma

This comment has been minimized.

Show comment
Hide comment
@singpolyma

singpolyma Feb 25, 2016

This would be very useful for compatibility with ledger. the complexity mostly would come from interpreting the expression language, yes?

singpolyma commented Feb 25, 2016

This would be very useful for compatibility with ledger. the complexity mostly would come from interpreting the expression language, yes?

@simonmichael

This comment has been minimized.

Show comment
Hide comment
@simonmichael

simonmichael Feb 25, 2016

Owner

Yes interpreting the expression language, for a start. That alone seems a really big task. Also there's: finding out what that language is (from the C++ code, as neither the docs nor any person knows all of it). Troubleshooting/fixing/documenting the inevitable incompatibilities in the features that have been implemented. Tracking the language as Ledger evolves over time. Troubleshooting/testing/fixing/documenting all the interesting interactions we'll find between automated transactions and existing features.

Arguably it would be useful to at least specify and implement a compatible subset of it. It sounds like it would be useful for some folks, and it would make us seem more Ledger compatible - right ? But most Ledger users that try it will grumble that it doesn't support essential-thing-they-rely on; hledger users will be constantly asking for just one more feature; and it will add complexity and drag to the project which I'm not sure it will justify, even if we manage to attract a dedicated maintainer for it.

Soo.. that's my reaction today, but I'd love to be proven wrong. How interested are you really ? :-)

Owner

simonmichael commented Feb 25, 2016

Yes interpreting the expression language, for a start. That alone seems a really big task. Also there's: finding out what that language is (from the C++ code, as neither the docs nor any person knows all of it). Troubleshooting/fixing/documenting the inevitable incompatibilities in the features that have been implemented. Tracking the language as Ledger evolves over time. Troubleshooting/testing/fixing/documenting all the interesting interactions we'll find between automated transactions and existing features.

Arguably it would be useful to at least specify and implement a compatible subset of it. It sounds like it would be useful for some folks, and it would make us seem more Ledger compatible - right ? But most Ledger users that try it will grumble that it doesn't support essential-thing-they-rely on; hledger users will be constantly asking for just one more feature; and it will add complexity and drag to the project which I'm not sure it will justify, even if we manage to attract a dedicated maintainer for it.

Soo.. that's my reaction today, but I'd love to be proven wrong. How interested are you really ? :-)

@simonmichael

This comment has been minimized.

Show comment
Hide comment
@simonmichael

simonmichael Feb 25, 2016

Owner

PS, I said "not sure it will justify" partly because I have philosophical/design doubts too, namely that generating "invisible transactions" adds a level of complexity and error potential for users that is completely absent from the current hledger, where everything is visible and explicit, the journal is readable by humans and accountants even without software, and we do everything we can to make things intuitive and minimise avoidable complexity.

All that said, if someone made it, I'd be first in line to try it and figure out how I can best use it. An add-on script might be a good way to start, for preprocessing journals with automated transactions.

Owner

simonmichael commented Feb 25, 2016

PS, I said "not sure it will justify" partly because I have philosophical/design doubts too, namely that generating "invisible transactions" adds a level of complexity and error potential for users that is completely absent from the current hledger, where everything is visible and explicit, the journal is readable by humans and accountants even without software, and we do everything we can to make things intuitive and minimise avoidable complexity.

All that said, if someone made it, I'd be first in line to try it and figure out how I can best use it. An add-on script might be a good way to start, for preprocessing journals with automated transactions.

@simonmichael

This comment has been minimized.

Show comment
Hide comment
@simonmichael

simonmichael Jan 5, 2017

Owner

The hledger-rewrite script is a way to experiment with this. It generates postings according to rules, but you run it explicitly as part of your command pipeline.

Owner

simonmichael commented Jan 5, 2017

The hledger-rewrite script is a way to experiment with this. It generates postings according to rules, but you run it explicitly as part of your command pipeline.

@ony

This comment has been minimized.

Show comment
Hide comment
@ony

ony Jan 5, 2017

Collaborator

@simonmichael, I've tried to use it for automatic budget bucketing. See budget.bash.
Unfortunately for ./budget.sh -f data/sample.journal it generates:

2008/06/03 * eat & shop
    expenses:food                   $1
    expenses:supplies               $1
    assets:cash                    $-2
    (Budget:Expenses:Food)           $-1

Instead of

2008/06/03 * eat & shop
    expenses:food                   $1
    expenses:supplies               $1
    assets:cash                    $-2
    (Budget:Expenses:Food)           $-1
    (Budget:Expenses:Supplies)           $-1

And of course when I remove all not:... that I tried to use to avoid assigning posting to multiple buckets:

2008/06/03 * eat & shop
    expenses:food                       $1
    expenses:supplies                   $1
    assets:cash                        $-2
    (Budget:Expenses:Food)             $-1
    (Budget:Expenses:Supplies)           $-1
    (Budget:Expenses)                  $-1
Collaborator

ony commented Jan 5, 2017

@simonmichael, I've tried to use it for automatic budget bucketing. See budget.bash.
Unfortunately for ./budget.sh -f data/sample.journal it generates:

2008/06/03 * eat & shop
    expenses:food                   $1
    expenses:supplies               $1
    assets:cash                    $-2
    (Budget:Expenses:Food)           $-1

Instead of

2008/06/03 * eat & shop
    expenses:food                   $1
    expenses:supplies               $1
    assets:cash                    $-2
    (Budget:Expenses:Food)           $-1
    (Budget:Expenses:Supplies)           $-1

And of course when I remove all not:... that I tried to use to avoid assigning posting to multiple buckets:

2008/06/03 * eat & shop
    expenses:food                       $1
    expenses:supplies                   $1
    assets:cash                        $-2
    (Budget:Expenses:Food)             $-1
    (Budget:Expenses:Supplies)           $-1
    (Budget:Expenses)                  $-1
@simonmichael

This comment has been minimized.

Show comment
Hide comment
@simonmichael

simonmichael Jan 5, 2017

Owner

That's a pretty fascinating example of dark bash magic!

Owner

simonmichael commented Jan 5, 2017

That's a pretty fascinating example of dark bash magic!

@simonmichael

This comment has been minimized.

Show comment
Hide comment
@simonmichael
Owner

simonmichael commented Jan 6, 2017

Noted some rewrite todos.

@simonmichael

This comment has been minimized.

Show comment
Hide comment
@simonmichael

simonmichael Jan 6, 2017

Owner

Here's my attempt with the current hledger-rewrite; clunky, but possible:

budget.sh

#!/bin/sh
cat "$@" \
| hledger-rewrite -f- expenses:food     --add-posting '(budget:food)      *-1' \
| hledger-rewrite -f- expenses:supplies --add-posting '(budget:supplies)  *-1' \
| hledger-rewrite -f- expenses:rent     --add-posting '(budget:rent)      *-1' \
$ budget.sh data/sample.journal
2008/01/01 income
    assets:bank:checking            $1
    income:salary                  $-1

2008/06/01 gift
    assets:bank:checking            $1
    income:gifts                   $-1

2008/06/02 save
    assets:bank:saving              $1
    assets:bank:checking           $-1

2008/06/03 * eat & shop
    expenses:food                $1
    expenses:supplies            $1
    assets:cash                 $-2
    (budget:food)               $-1
    (budget:supplies)           $-1

2008/12/31 * pay off
    liabilities:debts               $1
    assets:bank:checking           $-1

Owner

simonmichael commented Jan 6, 2017

Here's my attempt with the current hledger-rewrite; clunky, but possible:

budget.sh

#!/bin/sh
cat "$@" \
| hledger-rewrite -f- expenses:food     --add-posting '(budget:food)      *-1' \
| hledger-rewrite -f- expenses:supplies --add-posting '(budget:supplies)  *-1' \
| hledger-rewrite -f- expenses:rent     --add-posting '(budget:rent)      *-1' \
$ budget.sh data/sample.journal
2008/01/01 income
    assets:bank:checking            $1
    income:salary                  $-1

2008/06/01 gift
    assets:bank:checking            $1
    income:gifts                   $-1

2008/06/02 save
    assets:bank:saving              $1
    assets:bank:checking           $-1

2008/06/03 * eat & shop
    expenses:food                $1
    expenses:supplies            $1
    assets:cash                 $-2
    (budget:food)               $-1
    (budget:supplies)           $-1

2008/12/31 * pay off
    liabilities:debts               $1
    assets:bank:checking           $-1

@linuxcaffe

This comment has been minimized.

Show comment
Hide comment
@linuxcaffe

linuxcaffe Jan 6, 2017

clunky? brilliant!

linuxcaffe commented Jan 6, 2017

clunky? brilliant!

@ony

This comment has been minimized.

Show comment
Hide comment
@ony

ony Jan 6, 2017

Collaborator

@simonmichael, how you catch all other expenses that have no specific bucket in your budget? They are still expenses and you can't simply ignore them.
I thought I can target them via query expenses and not:expenses:food. But it didn't work out.

Collaborator

ony commented Jan 6, 2017

@simonmichael, how you catch all other expenses that have no specific bucket in your budget? They are still expenses and you can't simply ignore them.
I thought I can target them via query expenses and not:expenses:food. But it didn't work out.

@simonmichael

This comment has been minimized.

Show comment
Hide comment
@simonmichael

simonmichael Jan 6, 2017

Owner

Hmm, it's difficult. Easier to make hledger-rewrite smarter I think.

Owner

simonmichael commented Jan 6, 2017

Hmm, it's difficult. Easier to make hledger-rewrite smarter I think.

@ony

This comment has been minimized.

Show comment
Hide comment
@ony

ony Jan 13, 2017

Collaborator

@simonmichael, I've started work on this ticket in my branch.

Current progress (hledger rewrite)

  • Apply for every posting in transaction
  • Full syntax of posting (including comments etc).
  • ModifierTransactions from journal used for rewriting.
  • mtvaluexpr is parsed as Query but it shouldn't refer to a current date. It is used together with command-line query to restrict what transactions being re-written.

See changes to tests/bin/rewrite.test for examples of automated transactions.

Collaborator

ony commented Jan 13, 2017

@simonmichael, I've started work on this ticket in my branch.

Current progress (hledger rewrite)

  • Apply for every posting in transaction
  • Full syntax of posting (including comments etc).
  • ModifierTransactions from journal used for rewriting.
  • mtvaluexpr is parsed as Query but it shouldn't refer to a current date. It is used together with command-line query to restrict what transactions being re-written.

See changes to tests/bin/rewrite.test for examples of automated transactions.

ony added a commit to ony/hledger that referenced this issue Jan 13, 2017

ony added a commit to ony/hledger that referenced this issue Jan 13, 2017

ony added a commit to ony/hledger that referenced this issue Jan 13, 2017

ony added a commit to ony/hledger that referenced this issue Jan 13, 2017

ony added a commit to ony/hledger that referenced this issue Jan 13, 2017

simonmichael added a commit that referenced this issue Jan 13, 2017

Apply ModifierTransaction's from journal during hledger-rewrite. (#477)
* rewrite: rewrite every posting

Also start using ModifierTransaction

* rewrite: use journal parser for postings

* rewrite: use ModifierTransactions from Journal

See #99

mstksg added a commit to mstksg/hledger that referenced this issue Feb 3, 2017

Apply ModifierTransaction's from journal during hledger-rewrite. (sim…
…onmichael#477)

* rewrite: rewrite every posting

Also start using ModifierTransaction

* rewrite: use journal parser for postings

* rewrite: use ModifierTransactions from Journal

See simonmichael#99
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment