Report unrealized capital gains/losses

[simonmichael]

Part of #1015. Related: #82, #377, #1019. With --value-at (#329) It's now easier to see the changes in value due to market price changes over time. However we don't yet fully model those gains/losses/revaluations as transactions, so the accounting equation is not satisfied and balance report row totals look wrong.

Notes

what are some events of interest ?

• purchasing a priced commodity
• change in price of a priced commodity currently held/owed (recognise unrealised gain/loss)
• selling a priced commodity (& realise gain/loss)
• borrowing/lending a priced commodity
• repaying/recovering a priced commodity
• end of a reporting period

What is the relationship between "revaluation", recognising (unrealised) capital gains, and realising capital gains ?

I think:

• Revaluation and recognising unrealised capital gains(&losses), are the same operation.
• This operation isn't a concrete transaction appearing in the journal. It's a synthetic, report-time transaction, appearing only when commodities are being valued.
• It isn't an inflow or outflow. It affects the balance sheet. It can be modelled as a transfer between assets (or liabilities) and equity, eg equity:valuation or similar.
• It records the swelling(shrinking) effect of external changes (market price/exchange rate changes) on value, necessary to satisfy the accounting equation.
• It can be calculated "bottom up", by summing the value changes due to each individual lot, based on their amount, purchase costs and end values. Lot purchase dates don't really matter.
• Or it can be calculated "top down", by subtracting a balance's starting value plus posted changes, from its end value. The difference is the gain/loss due to all market price changes in the period. This is probably easier.
• Whether bottom-up or top-down, it can be calculated at different time granularities - eg at every change in market price, or once for a whole period - according to reporting needs.

While

• Realising capital gains is a concrete transaction, recorded in the journal.
• It accompanies another transaction which triggers it (eg, the selling of stock).
• It records the capturing of the unrealised gains.
• It is an inflow (gain) or outflow (loss). It affects the income statement. (Usually. There are several ways to record it.)

How could you report unrealised gains using current features ?

Are there any relevant, different procedures recommended in the Selinger multicurrency tutorial ?

When should unrealised gains appear in reports ?

Should gains be shown with all account types ? (revenues, expenses, equity.. )

[adept]

My $.02:

0. Unrealized gains are not necessarily related to lots: it could be that you care about current/historical value, but have no obligation to do LIFO/FIFO accounting of lots, so you dont need to track them (example: ISA accounts in UK).

1. Sometimes you might not know or you might not care about price. Example: your pension fund might send you quarterly statements about current value of your pension pot, and you are content to track just that, without recording prices of all constituent securities, their amounts etc. So, in my opinion, you could have "valuation" as the event, which is akin to change of price.

2. There might be value in explicitly recording unrealized gains/losses as transactions: it allows you to have several different PnL "heaps". For example, you might want to track your crypto + P2P lending gains/losses separately from everything else. It also allows you to easily "realize" the pnl via transaction that asserts that balance of unrealized PnL account is zero and everything is going into named second account (again, this is possible when you do not need to calculate/report Capital Gains tax etc and dont care about lots).

3. Realizing unrealized PnL and capital gains are not necessarily the same (though they often are). You can realize unrealized PnL by selling what you own, but it could be that capital gains (as a taxable event) could be deferred/prorated/offset/split between periods, so "capital gains" are not necessarily equal to realized PnL

If you dont care about lots, then you could just periodically record value of your holdings by entering transactions like:

YYYY-MM-DD valuation
asset:whatever = $(current-value)
equity:unrealized pnl

You can then use "hledger roi" (pointing it to these two accounts) to track performance of your holdings. The more valuations you have - the better.

[zoranzaric]

I've been tracking my lots manually like this: http://rantsideasstuff.com/posts/2018/07/01-tracking-investments-in-lots-with-hledger/

Something I haven't seen being talked about in this context are fees. Any thoughts on those?

[erikryb]

Here is my proposal for how this should be done.

First of all, we will require that all transactions in the ledger are balanced. Hledger currently checks this for transactions containing only one commodity, but not when multiple commodities are included in the transaction. One way to do this in a way that is backwards-compatible is to introduce a new account named UNBALANCED, and then to use this account to balance each transaction when generating reports. For instance, consider the following transaction:

2019-04-01 Buy stocks
    assets:cash          -$1000
    assets:investments  1 AAPL

This transaction is currently unbalanced, since the amounts sum up to -$1000, and 1 AAPL. When generating reports, this transaction would be turned into this:

2019-04-01 Buy stocks
    assets:cash         -$1000
    assets:investments  1 AAPL
    UNBALANCED           $1000
    UNBALANCED         -1 AAPL

An additional flag (--force-balanced or similar) could be introduced. When activated, this checks that all transactions are balanced, and if not gives an error message. To avoid this error message, the user would need to make explicit trading accounts, for instance as follows:

2019-04-01 Buy stocks
    assets:cash         -$1000
    assets:investments  1 AAPL
    trading              $1000
    trading            -1 AAPL

Now that all transactions are balanced, I will explain how the -V flag should affect each report. I will focus on what the reports should look like, and not how they are computed. The reports may be calculated in different ways, as long as the result stays the same.

Balance sheets

• The command bal -M -V -H should do exactly what bal -M -V -H --value-at=period currently does.
• The command bal -M -V should first compute bal -M -V -H --value-at=period, and then display the differences in balance for each period. The interpretation should be that this command shows the changes in balance for each period. This interpretation currently holds when -V is not used, and should also hold when -V is used.

All questions we may have about the performance of our investments may be answered by looking at the balances (historical or changes) of the trading account and the assets:investments account.

Print / register

The print / register commands show individual transactions. When -V is used, additional valuation transactions need to be generated to account for changes in valuations. This may be done as follows. First we convert all transaction postings to the base currency at the date of the transfer. For our example above, we would get the transaction.

2019-04-01 Buy stocks
    assets:cash         -$1000
    assets:investments   $1000
    trading              $1000
    trading             -$1000

The two trading postings add up to zero, so this transaction should be displayed as

2019-04-01 Buy stocks
    assets:cash         -$1000
    assets:investments   $1000

Now, for each commodity valuation, for instance P 2019-04-10 AAPL $1200, we add a transaction

2019-04-10 Valuation: AAPL $1200
    assets:investments   $200
    trading             -$200

This transaction involves all accounts that have a non-zero balance of AAPL, and corrects the balance of each account with the updated price. This may result in a lot of valuation transactions, so if we want to we may add a flag --price-frequency which takes a period (daily, weekly, monthly, quarterly, or yearly) as argument. When this flag is used, the marked prices are filtered so that there are a maximum of one commodity valuation per period.

Implementation steps

The proposal may be implemented in several steps:

1. Modify bal -V as described above so that it correctly shows the balance changes of the valued historical balances.
2. Ensure that all transactions are balanced as described above with the UNBALANCED account and the --force-balanced flag.
3. Implement valuation transactions for the print and register commands.

Closing remarks

• Step 2 is a prerequisite for step 3 to ensure that the valuation transactions are balanced.
• When all steps are implemented, running bal -M -V or bal -M -V -H should be exactly the same as running print -V first and then running bal -M or bal -M -H on the resulting journal.
• This proposal is simple and powerful. It treats all accounts equally and does not involve "special accounts".

[simonmichael]

@adept, I think we are generally in agreement.

Unrealized gains are not necessarily related to lots: it could be that
you care about current/historical value, but have no obligation to do
LIFO/FIFO accounting of lots, so you dont need to track them (example:
ISA accounts in UK).

Sometimes you might not know or you might not care about price.
Example: your pension fund might send you quarterly statements about
current value of your pension pot, and you are content to track just
that, without recording prices of all constituent securities, their
amounts etc. So, in my opinion, you could have "valuation" as the
event, which is akin to change of price.

Yes, unrealised gains could be calculated from individual lots, but
can also be calculated for whole balances, without considering lots.

There might be value in explicitly recording unrealized gains/losses
as transactions: it allows you to have several different PnL "heaps".
For example, you might want to track your crypto + P2P lending
gains/losses separately from everything else. It also allows you to
easily "realize" the pnl via transaction that asserts that balance of
unrealized PnL account is zero and everything is going into named
second account (again, this is possible when you do not need to
calculate/report Capital Gains tax etc and dont care about lots).

I don't quite follow.. I'm expecting unrealised gains to be calculated
in each account, so there should be no problem seeing them separately.

Maybe the term "Unrealized PnL" is a little confusing ? It's not on
the income statement yet. I think I'll stick to "unrealised gain/loss"
or "recognised gain/loss".

Realizing unrealized PnL and capital gains are not necessarily the
same (though they often are). You can realize unrealized PnL by
selling what you own, but it could be that capital gains (as a taxable
event) could be deferred/prorated/offset/split between periods, so
"capital gains" are not necessarily equal to realized PnL

I don't understand your terminology here. I wonder if you're pointing
to something like MisterY's stock split example:
#1022 (comment)
In any case, I'll try to keep discussion of realising capital gains
for a separate subissue of #1015 (#488).

[simonmichael]

@zoranzaric: what's the problem with fees ?

Thank you all for the thoughtful input. When I don't respond quickly it's because of limited time & brain space. There's an awful lot to think about in this space. :)

[simonmichael]

Maybe the term "Unrealized PnL" is a little confusing ?

Never mind, just getting my terms straight.

[zoranzaric]

@simonmichael: It would be helpful to have fees subtracted from unrealised gains to see my "real" gains.

[simonmichael]

First of all, we will require that all transactions in the ledger are
balanced. Hledger currently checks this for transactions containing
only one commodity, but not when multiple commodities are included in
the transaction. One way to do this in a way that is
backwards-compatible is to introduce a new account named UNBALANCED,
and then to use this account to balance each transaction when
generating reports. For instance, consider the following transaction:

2019-04-01 Buy stocks
assets:cash -$1000
assets:investments 1 AAPL

This transaction is currently unbalanced, since the amounts sum up to
-$1000, and 1 AAPL.

I guess Ledger/hledger use that word differently.. they consider the
above to be syntactic sugar for the following transaction, which they
call balanced:

2019-04-01 Buy stocks
    assets:cash                   -$1000
    assets:investments   1 AAPL @@ $1000

When generating reports, this transaction would be
turned into this:

2019-04-01 Buy stocks
assets:cash -$1000
assets:investments 1 AAPL
UNBALANCED $1000
UNBALANCED -1 AAPL

An additional flag (--force-balanced or similar) could be introduced.
When activated, this checks that all transactions are balanced, and if
not gives an error message. To avoid this error message, the user would need to make explicit
trading accounts, for instance as follows:

2019-04-01 Buy stocks
assets:cash -$1000
assets:investments 1 AAPL
trading $1000
trading -1 AAPL

Related to #758, "Treacherous multi-currency auto-inference".

Balance sheets

• The command bal -M -V -H should do exactly what bal -M -V -H
  --value-at=period currently does.

• The command bal -M -V should first compute bal -M -V -H
  --value-at=period, and then display the differences in balance for
  each period. The interpretation should be that this command shows
  the changes in balance for each period. This interpretation
  currently holds when -V is not used, and should also hold when -V is
  used.

All questions we may have about the performance of our investments may
be answered by looking at the balances (historical or changes) of the
trading account and the assets:investments account.

I'm too long away from the code to comment on this in detail; maybe later.

Print / register

The print / register commands show individual transactions. When -V is
used, additional valuation transactions need to be generated to
account for changes in valuations. This may be done as follows. First
we convert all transaction postings to the base currency at the date
of the transfer. For our example above, we would get the transaction.

2019-04-01 Buy stocks
assets:cash -$1000
assets:investments $1000
trading $1000
trading -$1000

This has been my original thought too - model revaluation in depth, ie as
transaction events, and let the reports do their thing. I called this
"bottom up" before.

Then there is the "top down", shortcut method where we would only
calculate the revaluations where needed, ie in balance reports. This
way the calculation is simpler, you calculate only for the accounts
and periods being shown in the balance report.

Calculating for every account, depending on which commodities it
holds, potentially at every price change, sounds like a lot of work.
Though, it might be that the two methods work out equally efficient
due to laziness.

Including revaluations in balance and register reports is necessary to
keep the accounting equation satisfied. Showing revaluation events in
print reports isn't strictly necessary, though it does sound nice for
consistency, understanding, and troubleshooting.

The two trading postings add up to zero, so this transaction should be
displayed as

2019-04-01 Buy stocks
assets:cash -$1000
assets:investments $1000

Hiding postings written explicitly in the journal would be quite a new
behaviour for print.

Now, for each commodity valuation, for instance P 2019-04-10 AAPL
$1200, we add a transaction

2019-04-10 Valuation: AAPL $1200
assets:investments $200
trading -$200

This transaction involves all accounts that have a non-zero balance of
AAPL, and corrects the balance of each account with the updated price.
This may result in a lot of valuation transactions, so if we want to
we may add a flag --price-frequency which takes a period (daily,
weekly, monthly, quarterly, or yearly) as argument. When this flag is
used, the marked prices are filtered so that there are a maximum of
one commodity valuation per period.

In non-periodic balance reports, one revaluation for the whole report
period is enough.

In periodic register/balance reports, the periods are a natural
interval for revaluations.

In print reports and non-periodic register reports, there's no
obvious frequency or date at which to insert revaluations. It could be
as specified by a new flag as you say, or once at the end, or (in the
case of print) as specified by -D/W/M/Q/Y.

[erikryb]

Thanks for the reply.

I guess Ledger/hledger use that word differently.. they consider the
above to be syntactic sugar for the following transaction, which they
call balanced:

2019-04-01 Buy stocks
    assets:cash                   -$1000
    assets:investments   1 AAPL @@ $1000

Yes, this is a mistake in my opinion. If we allow such transactions, then it becomes possible to create money out of nowhere, as explained here. If we want to add valuation transactions the way I described, all transactions must be balanced for each commodity.

This has been my original thought too - model revaluation in depth, ie as
transaction events, and let the reports do their thing. I called this
"bottom up" before.

Then there is the "top down", shortcut method where we would only
calculate the revaluations where needed, ie in balance reports. This
way the calculation is simpler, you calculate only for the accounts
and periods being shown in the balance report.

Calculating for every account, depending on which commodities it
holds, potentially at every price change, sounds like a lot of work.
Though, it might be that the two methods work out equally efficient
due to laziness.

Yes, there are performance issues that must be considered here. I would guess that the implicit top down method would be faster for balance reports, but for register and print reports we need explicit valuation transactions (bottom up).

Including revaluations in balance and register reports is necessary to
keep the accounting equation satisfied. Showing revaluation events in
print reports isn't strictly necessary, though it does sound nice for
consistency, understanding, and troubleshooting.

Having revaluation transactions in print would indeed be good for consistency. It would be nice if

hledger print -V | hledger -f - balance -M

gave the same output as

hledger balance -MV.

Hiding postings written explicitly in the journal would be quite a new
behaviour for print.

Yes, this is not strictly necessary. Perhaps we could add a normalization option to print if we want to.

In non-periodic balance reports, one revaluation for the whole report
period is enough.

In periodic register/balance reports, the periods are a natural
interval for revaluations.

Agreed!

In print reports and non-periodic register reports, there's no
obvious frequency or date at which to insert revaluations. It could be
as specified by a new flag as you say, or once at the end, or (in the
case of print) as specified by -D/W/M/Q/Y.

My preference would be a new flag, with daily revaluations being the default. Note that there would not be any revaluations in the periods where there are no price directives, so the number of revaluations are limited to the number of price directives.

[danieldjewell]

I guess something to possibly keep in mind on this would be tracking things like automatic dividend reinvestment (e.g. dividend paid out, converted to a stock purchase >> shares). This gets a little tricky because, over time, you could have many tiny "purchases" (gotta love owning like 1/10000 of a share in a company 😁...) ... each of these purchases has it's own cost basis and it's own unrealized gain/loss. (And it's own Capital Gains loss position long/short).

It really isn't much different to having multiple purchases on different dates at different prices -- you could buy 100 MSFT on 01-JAN @ $10.00 and another 100 MSFT on 01-FEB @ $20.00 ... if today is 01-MAR and MSFT is @ $30.00, (1: 100x10=$1000 cost basis; current= $3000; 2: 100x20=2000; current =$3000; total unrealized = $3000 ...)

[alensiljak]

Revaluation and recognising unrealised capital gains(&losses), are the same operation.

Not sure about accounting for fixed assets but sometimes these are explicitly acknowledged in the books with a transaction, i.e. once a year.
But with stocks, this does not happen. Their current market value is calculated in comparison to the current market price and this seems the most common scenario. It would be too much (and quite pointless) too revalue stock holdings explicitly, considering that the current price is just an information and not necessarily what we would get in a sale.
For both cases it seems that an in-memory calculation provided by software would be sufficient.

[alensiljak]

@adept,

Unrealized gains are not necessarily related to lots

Quite the contrary, I think un/realized gains/losses can't even be calculated without lots. More about how I see lots, here: #1022 (comment)

[adept]

I am going to re-read everything that happened on #1022 recently, but I am going to strongly disagree with your statement even before I've done that :)

Let's say I am buying stock and I am recording this as transactions like this one:

yyyy-mm-dd buying stocks
  assets:checking -$100
  assets:investment

(notice how I am not even recording WHICH stocks I bought or the current price).

Let's say I am doing this every month, so at the end of the year by checking accoung is down $1200 and my investment account is up $1200.

At this point I get current valuation of investment account (by going online, receiving statement, etc) and it is $1234 at this point. So i record:

yyyy-mm-dd Current investment valuation
  account:investment   = $1234
  assets:unrealized pnl

Right there, my investment account has correct balance, and "unrealized pnl" account has $34 of unrealized PnL. Yet I recorded no prices, no lots, no stock tickers etc.

If your taxation/accounting regime does not require you to do LIFO/FIFO accounting of lots, it is perfectly feasible.

[adept]

To address this point : "But with stocks, this does not happen. Their current market value is calculated in comparison to the current market price and this seems the most common scenario. It would be too much (and quite pointless) too revalue stock holdings explicitly, considering that the current price is just an information and not necessarily what we would get in a sale."

I am going to make another bold claim here: for a person investing in stocks (long-term) and not day-trading them daily fluctuations in price are always meaningless and it is perfectly fine to get somewhat-accurate valuations of the total of your holdings from some sort of benchmar source (be it Yahoo prices, Bloomberg prices, or a report from your brokerage even if you dont know which prices they are going to use). As long as you are using the same valuation report source consistently, it would not matter how precise your prices are (provided that they are reasonably precise), how do they lag behind the market, are they open or close prices, etc. For the duration of typical investment (years) none of this would matter, and precisely tracking your share lots and prices of shares would just create unnecessary burden for you.

Now, if you need this information for other purposes as well (like FIFO taxation) - fine, thats completely different story. But I claim that you should not be tracking or recording stock prices in a typical investment scenario otherwise, unless you have copious amounts of free time.

[alensiljak]

For the duration of typical investment (years) none of this would matter

Unfortunately it does unless you live in a perfect world. Mutual funds and agents go out of business, brokerage accounts change owners and rules, companies disappear, ETFs stop trading in certain markets, regulation changes, people move states and continents, citizenships and tax residence, and so on and so forth.
All this requires the owner to switch accounts, having often to sell shares, and report all the differences to the authorities.
I also wish life was simpler but, unfortunately, that doesn't seem to make any difference.

[adept]

All this requires the owner to switch accounts, having often to sell shares, and report all the differences to the authorities.

We seem to circle back to the "but you also need this for taxes" argument. I do agree that if you have to do fifo capital gains, you absolutely need lots.

But I think it is a mistake to think that it is a general requirement. It is in US (as far as I know), but for example in the UK you could happily invest in stocks without having any need to report capital gains or losses, and tracking lots would be just extra burden for you.

Title of this ticket is "unrealized gain/losses", and all my arguments are focused on not equating this to "tracking lots for tax purposes" (which is a valid and related goal, but IMO separate one)

[alensiljak]

Title of this ticket is "unrealized gain/losses", and all my arguments are focused on not equating this to "tracking lots for tax purposes"

How would you envision tracking unrealized gains for standard stocks and mutual funds without lots?
The fact that, in the mentioned case, the fund is doing it for you could be stretched to say that hledger doesn't need to calculate anything because one's bank, credit union, broker, retirement fund, etc. provide statements with the balances. I have a similar case with Vanguard mutual funds but with stock and ETF holdings it is not so simple. Someone still needs the lot information to calculate the values, irrespective of the taxes.

[adept]

Why ("current valuation" - "amount invested" ) would not be applicable to common stocks and mutual funds?

[alensiljak]

That's what it is. The question is just how you get to "current valuation".

[adept]

In my experience it is usually included in paper or electronic on-demand statement of one's broker.

[alensiljak]

Good. So, now that solution is limited to institutions that provide accurate financial statements with current balances. We can then forget about my sugar example, where we bought 18 units (tons?) of sugar at A$2.39 per kg. There is no financial institution that will provide us with the statement of the current market value, so we are dead in the water.
Also, if we happen to trade sugar regularly, we will end up in a situation where we have A$ 356,000 spent on our storage. When we read that the price of sugar dropped (or jumped) by A$ 1.25, we will not know how it affects us exactly.
And, if we made 23 purchases so far and sell A$ 27,368 worth of it, what exactly is our profit/loss? More importantly, what is the profit/loss the next time we sell A$ 112,514 worth of it if we have in the meantime also made another 6 purchases worth A$ 63,824.13?

I once worked at a company that was trading natural gas and electricity. If there was no accurate information about the potential profit, how would they even operate?

Probably no need to mention the case of shares which trade on multiple exchanges, in multiple currencies. Ledger also can't handle these.

Taxation is not the only reason to have an accurate picture of what's going on with our finances or other commodities.
Rebalancing of the portfolio (risk management) is another important factor. How do we know our gold allocation is significantly higher to where we want it to be (which it is right now)?

What I'm saying, among other things, is that, if you have accurate statements already, why would you need hledger?

Edit: corrections in spelling and meaning.

[adept]

Well, it looks to me like we are in some sort of violent agreement here :)

You are arguing that there are scenarios where having precise lots information is highly beneficial (for the purposes of computing/tracking gains/losses and for other purposes as well). In your sugar example you would obviously be better off tracking lots

I am arguing that, on the other hand, there are practical scenarios where tracking gains/losses could be achieved without tracking lots, and therefore it is not prudent to say that "you can never ever do gains/losses without tracking lots". In my "casual investor" example you would be better off NOT tracking lots.

Furthermore, tracking lots is not cost-free (forgive me the pun). As soon as you start tracking lots, you have to maintain information about lot quantites. To go back to shares example, you now commit yourself to tracking forward and reverse stock spilts, rights and warrants issue, mergers and acquisitions, secondary offerings and other things that potentially change quantity of shares that you have without you actively trading them. If all you want is gains/losses, and you have access to information about wholesale value of your portfolio (from you broker, for example), then you will save yourself quite a lot of work by not tracking quantities.

This most recent chain of comments have started with "I think un/realized gains/losses can't even be calculated without lots". Do you still firmly believe that this is the case, or do you see some merit in my arguments that dependency between these two things is not quite as strong?

[alensiljak]

That summarizes it well, I'd say. :)
You are right in that having lot information is not always necessary. But the institution providing the statements is still using it, it's just not that obvious. It does come down to the level of detail one currently has vs wants to have, and effort in getting there. As always.

[simonmichael]

Thanks for the input. I (still) agree with Dmitry: you need to track lots only for calculating realized gains. Not for calculating unrealized gains.

[alensiljak]

In that case I did not properly understand the following explanation:

Or it can be calculated "top down", by subtracting a balance's starting value plus posted changes, from its end value. The difference is the gain/loss due to all market price changes in the period. This is probably easier.

Where is the "end value" coming from? Is this something we have to enter manually and does it replace/ignore the Price records?

[adept]

In technical terms, it could come from some form of a balance assertion.

[simonmichael]

“End value” there is the market value, based on market prices, which we assume are available. These are a requirement for any calculation of “capital gains”, unless I’m missing something ?

[alensiljak]

Ah, the balance expressed in the non-currency commodity, yes. Times the last price.

[alensiljak]

What happens with share transfers, gifts, dividend participation programs, etc. where there is no explicit value on the record?

[the-solipsist]

Thanks for the input. I (still) agree with Dmitry: you need to track lots only for calculating realized gains. Not for calculating unrealized gains.

I completely agree. Another way of putting it is: "Lots only become important when you sell a commodity for a profit/loss." Until that point of sale, your (unrealized) profit or loss only depends on the aggregate difference between the amount spent to acquire the commodity and the current market price of the commodity (which can even be a guess, and need not be exact), and not on the lots.

One way of simulating lots without any additional software support is with dated sub-accounts.

What happens with share transfers, gifts, dividend participation programs, etc. where there is no explicit value on the record?

Depending on the purpose of your tracking (taxation, self-knowledge, etc.), you have to provide a value that is acceptable for that purpose, whether it is a guess or derived through some method. For instance, tax authorities might require you to follow a particular method of valuation, and so forth. All of cost accountancy is about such valuation. There are guides on how to arrive at the fair value.

[the-solipsist]

I completely agree. Another way of putting it is: "Lots only become important when you sell a commodity for a profit/loss." Until that point of sale, your (unrealized) profit or loss only depends on the aggregate difference between the amount spent to acquire the commodity and the current market price of the commodity (which can even be a guess, and need not be exact), and not on the lots.

I would add that as I see it, lots don't become important unless a non-averaging cost basis (such as FIFO/LIFO/High Cost/Low Cost) is used. Because if you use average cost basis, then you don't need to specify which precise units of the commodity (bought on which date at which price) were sold.

This, for instance, is usually the case with currency conversions for the purpose of purchasing (as opposed to forex investment). If you're going on a trip from India to the USA and are converting from INR to USD multiple times, and then converting back, the gain/loss you'd look at is the difference between the weighted average cost of all purchases of USD and the final sale of the remaining USD back into INR (and subtracting conversion fees). Unlike in the case of investments, you wouldn't use FIFO, etc., and thus would not need to track the currencies you've bought and sold in the form of lots.

[pkel]

I'm new here. I have not completely parsed #1022 yet. Please bear with me if the following is redundant, inaccurate or insane.

lots don't become important unless a non-averaging cost basis (such as FIFO/LIFO/High Cost/Low Cost) is used.

I go a step further and claim that the recording of lots is only important if you want to do something special. As long as you stick to one of the standard deterministic methods, you do not have to track your lots. The software can do it for you, e.g., by maintaining a stack or queue on the account level.

Let me propose a something radical: a new equity account type which I will call T(rading) for now. T-accounts will track losses/gains due to change in valuation. With signed account values, the new bookkeeping equation is A + L = E = X + R + T. T-accounts add a few features on top of E-accounts.

1. T-accounts have an accounting method assigned (FIFO/LIFO/etc). I do LIFO here because I like stacks. Other methods have their own logic. Declaration might look like this.

   account trading  ; type:T, method:LIFO
   

2. T-accounts change the interpretation of multi-currency postings. Consider this record.

   2020-02-01 Tx 1
   assets:cash  -1000$
   assets:gme  1 GME
   trading
   

   Currently this is interpreted as:

   2020-02-01 Tx 1 (currently)
   assets:cash  -1000$
   assets:gme  1 GME @@ 1000$
   trading  0
   

   I propose to infer this interpretation (only for trading accounts, obviously):

   2020-02-01 Tx 1 (proposed)
   assets:cash  -1000$
   assets:gme    2 GME
   trading  1000$
   trading  -2 GME
   

3. Hledger infers that this was a buy and pushes 2GME/500$ to an internal stack associated with the trading account. (Do we need a base currency to infer buy/sell?)
4. For a sell I will enter the following.

   2020-02-09 Tx 2
   assets:cash  50$
   assets:gme  -1 GME
   trading
   

   As in 2., hledger extends this to

   2020-02-09 Tx 2 (proposed)
   assets:cash  50$
   assets:gme  -1 GME
   trading  -50$
   trading  1 GME
   

5. Hledger infers that this is a sell from the flipped signs, pops 2GME/500$, pushes 1GME/500$. At this point hledger knows that Tx 2 realizes a loss of 450$. With the above input, the realized loss would stay in the trading account. A slight modification of Tx 2 realizes the loss into another account:

   2020-02-09 Tx 3
   assets:cash  50$
   assets:gme  -1 GME
   trading ; realize into:"expenses:realized losses"
   

   becomes

   2020-02-09 Tx 3 (proposed)
   assets:cash  50$
   assets:gme   -1 GME
   trading  -50$
   trading  1 GME
   trading  -450$
   expenses:realized losses  450$
   

6. A default for realize into: can be set on account declaration.

Pros:

• I think this ensures that both unrealized and realized gains are always reported as equity when doing a bal -V or similar.
• no lot-accounts used, thus less verbose reports
• minimal manual input, thus less errors/frustration
• I think it can be compatible with old hledger, ledger and manually added lots.

Cons:

• I do not like the realize into:"expenses:realized losses" part. Accounts in tags seem awkward.

What do you think?

[pkel]

I think we can get rid of the realize into: tag.

1. unchanged.
2. unchanged.
3. unchanged.
4. For a sell I will enter the following.

   2020-02-09 Tx 2b
   assets:cash  50$
   assets:gme  -1 GME
   trading
   realized
   

   As Unlike in 2., hledger extends this to

   2020-02-09 Tx 2b (proposed)
   assets:cash  50$
   assets:gme  -1 GME
   trading  -500$
   trading  1 GME
   realized
   

   using the cost basis at the top of the stack. -450$ realized losses will be transferred to realized following the existing inference rules. Hledger will complain "unbalanced transaction" if no account is given for the gains/losses.

[simonmichael]

I'm way behind on this issue, is someone able to summarise rather briefly the current status and some reasonable next goal & actions ? Does balance --gain resolve it ?

[pkel]

There was an interesting discussion about whether reporting unrealized gains requires tracking lots. I think the conclusion was that basic reporting can be achieved without lots but in order to handle all edge-cases we have to track lots. balance --gain achieves the former but not the latter.

My previous two messages in this thread are about automatic lotting with predefined rules like FIFO or LIFO. I think the recent hledger updates go into the proposed direction.

1. account tags could be used to specify the accounting method for the trading account(s)
2. --infer-equity could be extended to create/consume lots according to the accounting method of the equity account

It might be better to continue the lot tracking discussion in #1022 though.

[alensiljak]

Does balance --gain resolve it ?

Indeed, balance --gain seems to do the job. It takes the initial value and the price changes into account. I have tried with a simple common scenario of purchasing a stock and having price changes. Also, selling a part of the holdings correctly indicates the values.

[alensiljak]

I believe I understand this case a bit better but I'm not sure that

yyyy-mm-dd Current investment valuation
  account:investment   = $1234
  assets:unrealized pnl

would work right now. It is inferring twice. Technically, it could work by first ordering the calculations.

I think that explicitly using the assets:unrealized pnl account is not what is implied by "report unrealized capital gains/losses". The way you are suggesting is manual and requires (almost) no calculation on the side of hledger. If the transaction above does not work right away, it is fairly easy to calculate the difference amount with a calculator by subtracting the previously-known balance (h bal account:investment) and book the transaction as:

yyyy-mm-dd Current investment valuation
  account:investment   = $1234
  assets:unrealized pnl  -$25

The case is pretty limited, though. Because it completely ignores the cases which involve the current market valuation based on price information. It does work for the case you suggest, in any case, where the valuation is calculated by a 3rd party.

What I assume by reporting unrealized capital gains/losses is what bal --gain does, taking into consideration changes in valuation due to market price chages (P records), as Simon suggests. The assets:unrealized pnl above should be a virtual account, created only for the purpose of showing the unrealized difference in valuation for the report/query.
Hledger puts the amount in the actual account.

> h bal --gain
        -2182.05 AUD  Assets:Investments:Shares:XYZ
--------------------
        -2182.05 AUD

Ledger lists it in Equity, for example.

> l b --unrealized ass and xyz -X AUD
        1,098.30 AUD  Assets:Investments:Shares:XYZ
        3,912.04 AUD  Equity:Unrealized Losses
--------------------
        5,010.34 AUD

But it is not stored in the book anywhere.

I am going to try the conversion of the portfolio and see how this works with my data.

[alensiljak]

Some reference: https://www.investopedia.com/terms/u/unrealizedgain.asp

[the-solipsist]

Hi, @alensiljak. I'm not quite sure what you're trying to do. There are two ways of showing what the Unrealized P/L is. The first is via a dynamic report (such as bal --gain), which is based on per-unit price declarations. The second is by adding an explicit entry in one's journal, which one would have to periodically update, using valuations (rather than per-unit price declarations).

They are just two different ways of seeing the same thing.

You could do:

P 2022-11-11 USD INR 82.00
$ hledger bal --gain

or you could do

2022-11-11 Update valuation
  account:investments:usd        = INR 1234.00  ; calculated at INR 82.00 per USD
  equity:unrealized-pnl

In the latter case, one would denominate account:investment in INR, and in the former, one would denominate it in USD.

You ought not be trying to mix both.

Having said that, I think the only thing left in this issue is "how best to display gains/losses: as part of the 'parent' account or as a separate Equity:Unrealized-P/L account?" The rest of this issue is more or less done thanks to bal --gain.