New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

HEP: clarify status field functionality, UI, docs #564

Closed
simonmichael opened this Issue Jun 5, 2017 · 1 comment

Comments

Projects
None yet
1 participant
@simonmichael
Owner

simonmichael commented Jun 5, 2017

The status field has been causing some confusion, and we have been brainstorming ways to improve its docs, querying UI, implementation etc. See related discussion links below. It seems that something should be done, but what ? Add more features ? Rework the internals ? Adopt new terminology ? Clarify the docs ? Accept things as they are and stop making a fuss ?

Here's the proposal I currently favour, being drafted in HEP format for more deliberation.

What is being proposed, in one sentence ?

Stop matching pending with --uncleared, add -P for --pending, and allow -U/-P/-C to be combined, making our existing UI more expressive, consistent and intuitive, at the cost of a small incompatibility and slightly more complex behaviour.

What real problem is being solved ?

Empirically, it's confusing to learn and teach how the status field works, what the status characters mean, how to filter precisely based on status, for at least some users.

Some ways of using the status field are inconvenient. Eg including only pending things in hledger-ui, or excluding pending things in the CLI (you must switch to using a status: query argument). The status: query (matching empty status field) does not work (#553). Pending status on postings is not displayed by hledger print or hledger-ui (#563).

Discussing, documenting and implementing these things takes too much time and effort, because of unclear (sic) terminology, in particular the difference between uncleared (unmarked) status and --uncleared ("not cleared") flag.

What is the proposed UI and help, if applicable ?

This proposal bundles several related changes, most important first. 4, 5, and 6 are the most optional:

  1. make --uncleared/-U match the uncleared (unmarked) status only. (Removes a cause of confusion, adds consistency. Con: reduces command line compatibility with Ledger; may trip up Ledger/hledger users used to the old behaviour.)

  2. make --pending more convenient: add a -P short flag and a P toggle key in hledger-ui. (Adds convenience and consistency. Con: may trip up Ledger users expecting -P to be --by-payee.)

  3. make the -U/-P/-C flags combinable, eg -UP, -PC, -UC. -UPC is equivalent to no flags. -UP is equivalent to the old -U behaviour. (Adds convenience, easier than using status:. Con: reduces command line compatibility with Ledger.)

  4. make hledger-ui's U/P/C toggles combinable. (Consistent with 3, adds power, more convenient than using status:. Con: adds complexity - 7 states instead of 3, harder to reset (just pressing U or C twice isn't enough, you have to look at the mode display and press the right keys, or use ESC which is a heavy hammer.))

  5. display the status mark for each transaction in hledger-ui register screen. (Provides an extra cue about the current status filtering mode. Con: may not be the best use of 2 columns in narrow windows.)

  6. display the status mark for each posting in hledger register report. (Provides an extra cue about what you're seeing, consistent with 5. Con: may be a waste of 2 columns, differs from Ledger register output.)

  7. start using "mark" as standard terminology for the character in the status field instead of "flag". (Avoids confusing it with the command-line flags in discussions.)

Help mockups:

  -U --uncleared          include only uncleared postings/txns
  -P --pending            include only pending postings/txns
  -C --cleared            include only cleared postings/txns
       -U --uncleared
              include only uncleared postings/txns.
              (-U, -P and -C may be combined).

       -P --pending
              include only pending postings/txns

       -C --cleared
              include only cleared postings/txns
       status:, status:!, status:*
              match  uncleared,  pending,  or   cleared   transactions
              respectively
│                                    U toggle uncleared/all                    │
│                                    P toggle pending/all                      │
│                                    C toggle cleared/all                      │

or

│                                    U show/hide uncleared                    │
│                                    P show/hide pending                      │
│                                    C show/hide cleared                      │

What is a rough draft of the reference documentation, or other details ?

Journal format:

Status

More about the status field: transactions, or individual postings
within a transaction, can be in one of three states, marked by a
single character preceding (and separated by a space from) the account name:

  • no status mark = uncleared
  • ! = pending
  • * = cleared

When reporting, you can filter by status using the status: query,
or the -U/--uncleared, -P/--pending, and -C/--cleared flags
(or the U, P, C keys in hledger-ui).

Note:
in hledger < 1.2 (and in Ledger still), -U meant "not cleared", and included pending.
But in hledger 1.3+, -U means uncleared (no status mark) only,
and you can combine status flags. Eg to match both uncleared and pending, use -UP (or not:status:*)
[More about this change].

Status marks are optional, but can be helpful eg for reconciling with real-world accounts.
Some editor modes provide highlighting and shortcuts working with status.
Eg in Emacs ledger-mode, you can toggle transaction status with C-c C-e, or posting status with C-c C-c.

What "uncleared", "pending", and "cleared" actually mean is up to you.
Here's one suggestion:

  • uncleared: recorded but not yet reconciled; needs checking
  • pending: during reconciliation, if needed: tentatively reconciled ("I think I have matched this one")
  • cleared: complete, reconciled/checked as far as possible, and considered correct.

With this scheme,
--cleared (or status:*) will show the current balance at your bank,
--uncleared (or status:) will show things which will probably hit your bank soon (eg uncashed checks),
and with neither flag (the default) you'll see the most up-to-date state of your finances.

hledger-ui:

U toggles whether transactions with uncleared status are shown.
P toggles whether transactions with pending status are shown.
C toggles whether transactions with cleared status are shown.
These modes can be combined; active status filters will be displayed in the mode line at top of window.
More about status.

How will this interact with existing features and systems ?

When users and command examples move between Ledger/hledger 1.2- and hledger 1.3+, -U should be translated to and from -UP. This might affect only users of pending status, and only in some cases.

hledger and hledger-ui are discussed in this proposal, but not hledger-web, hledger-api, or other addons. These should pick up the new flags from hledger-lib in their next release, and otherwise shouldn't be much affected.

How will we test or ensure this remains effective ?

Functional tests: tests/journal/status.test, tests/misc/query-status.test.

What is in and out of scope ?

The changes listed above. Bugs will be fixed separately.

What is an estimate of developer hours required ?

20

Which of our current Projects does this relate to ?

Discussion

@simonmichael simonmichael changed the title from clarify status field functionality, UI, docs to HEP: clarify status field functionality, UI, docs Jun 5, 2017

simonmichael added a commit that referenced this issue Jun 10, 2017

lib, cli: -U/--uncleared no longer matches pending things (#564)
Also begin using "marked"/"unmarked" terminology where it's helpful.

simonmichael added a commit that referenced this issue Jun 10, 2017

lib: multiple status: query terms are OR'd (#564)
Like desc: and acct:. I think this is more intuitive and useful,
so now eg "status: status:!" works (equivalent to -UP or "not:status:*").

@simonmichael simonmichael self-assigned this Jun 10, 2017

simonmichael added a commit that referenced this issue Jun 15, 2017

lib, cli: -U/--uncleared no longer matches pending things (#564)
Also begin using "marked"/"unmarked" terminology where it's helpful.

simonmichael added a commit that referenced this issue Jun 15, 2017

lib: multiple status: query terms are OR'd (#564)
Like desc: and acct:. I think this is more intuitive and useful,
so now eg "status: status:!" works (equivalent to -UP or "not:status:*").

simonmichael added a commit that referenced this issue Jun 15, 2017

rename "uncleared" status to "unmarked" and --uncleared to --unmarked (
…#564)

See the issue and linked mail list discussion. Ambiguity between the
uncleared state, and the "not cleared" --uncleared flag causes confusion
and friction. At this point it seems best to break with Ledger and
past hledger, pick a new name and drop --uncleared to put an end to it.

simonmichael added a commit that referenced this issue Jun 16, 2017

lib, cli: -U/--uncleared no longer matches pending things (#564)
Also begin using "marked"/"unmarked" terminology where it's helpful.

simonmichael added a commit that referenced this issue Jun 16, 2017

lib: multiple status: query terms are OR'd (#564)
Like desc: and acct:. I think this is more intuitive and useful,
so now eg "status: status:!" works (equivalent to -UP or "not:status:*").

simonmichael added a commit that referenced this issue Jun 16, 2017

rename "uncleared" status to "unmarked" and --uncleared to --unmarked (
…#564)

See the issue and linked mail list discussion. Ambiguity between the
uncleared state, and the "not cleared" --uncleared flag causes confusion
and friction. At this point it seems best to break with Ledger and
past hledger, pick a new name and drop --uncleared to put an end to it.
@simonmichael

This comment has been minimized.

Show comment
Hide comment
@simonmichael

simonmichael Jun 16, 2017

Owner

This has landed in master. Also multiple status filters in hledger-ui. The last is fiddly as expected..
Please build and use master as much as possible, to see if we like these changes. The next major release is in about two weeks, IIRC.

Owner

simonmichael commented Jun 16, 2017

This has landed in master. Also multiple status filters in hledger-ui. The last is fiddly as expected..
Please build and use master as much as possible, to see if we like these changes. The next major release is in about two weeks, IIRC.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment