add treefactor

[cyberthal]

Brief summary of what the package does

Treesort rapidly sorts text and files into outlines and the directory tree.

Direct link to the package repository

https://github.com/cyberthal/treesort

Your association with the package

I am the maintainer.

Relevant communications with the upstream package maintainer

None needed

Checklist

Please confirm with x:

• The package is released under a GPL-Compatible Free Software License.
• I've read CONTRIBUTING.org
• I've used the latest version of package-lint to check for packaging issues, and addressed its feedback
• My elisp byte-compiles cleanly
• M-x checkdoc is happy with my docstrings
• I've built and installed the package using the instructions in CONTRIBUTING.org
• I have confirmed some of these without doing them

[alphapapa]

This looks interesting, but it's very hard for me to figure out what it does.

• It appears related to Org-mode, but the mention of Org-related files and outlines is buried in the commentary.
• The name treesort suggests a library used to sort binary trees, but the functionality seems to be about manipulating outline headings and files.
• The package name trs is even less descriptive.
• The verb "throw" is used 22 times in the commentary. I feel like "throw" should be part of the package's name, since that seems to be its primary function--whatever "throwing" is, exactly ;).
• This sounds like data-loss, which is scary to a potential user:

trs-throw only imposes this opinion on you in one way: it creates Inbox.org files with a “*** offset” at the top. You can still file level-5 headings, but they might “vanish” if you file a level-4 heading that unintentionally folds appended level-5 headings beneath it.

I recommend taking some time to write some tests to ensure that the functionality you intend works properly, so that you don't accidentally delete user data.

• These kind of warnings sound inflexible:

When you throw text to an outline, trs-throw believes that the parent heading is at the top of the visible region. It will only throw to direct children of the parent. You should narrow appropriately before throwing.

trs-throw assumes that most headings you file will have four stars. Why? Imagine you are throwing headings to an outline. The level-1 heading is the document title. The level-2 headings are categories. The level-3 headings are sub-categories. The level-4 headings are topics. Outlines become unwieldy when they get too deep, at which point it’s better to create more files and directories to spread the load.

trs-throw only imposes this opinion on you in one way: it creates Inbox.org files with a “*** offset” at the top. You can still file level-5 headings, but they might “vanish” if you file a level-4 heading that unintentionally folds appended level-5 headings beneath it. You can also file level-3 headings, although they won’t be children of the “offset” heading, and might unexpectedly fold appended level-4 headings. I recommend that you convert headings to level 4 for transport, and then resize them at their destination.

It sounds kind of like your personal, idiosyncratic workflow, which is not likely to generalize to most other users.

• Treesort filing is fast. Think with your fingers, not your brain. You can always redo it later.

  That sounds interesting, but it's buried near the bottom, and it really needs a demonstration.

• Also, a general note about Org: this package seems to suggest using lots of small Org files in a deep hierarchy. As you note in the commentary, that's not helpful for Org Agenda. It's also not helpful for other Org tools, e.g. built-in searching tools like sparse trees, tag view, search view, etc, third-party tools like org-rifle and org-ql, and so forth. It can work, but it's much less efficient. Org can handle outline files several megabytes in size without difficulty (which is a lot of text), so deep hierarchies of small files are not generally recommended.

About the code:

• It's not idiomatic elisp, e.g. there are lots and lots of hanging parens. Please see https://github.com/bbatsov/emacs-lisp-style-guide

[cyberthal]

I'll add some gif demos.

I assume MELPA prefers the Outshine outline to be removed as well?

Since "Treesort" doesn't work, how about "Treefactor"? The goal is to streamline refactoring prose into a meta-outline composed of the directory hierarchy and text files containing outlines. One could merely use it for Dired filing, which is why I didn't present it as an Org package.

I use Treefactor to manage a daily intake of 1-10k words into a 600 mb git repo. That breaks down to maybe 10-100 headings daily. It doesn't delete user data. There are checks to prevent that. Are you suggesting writing unit tests? Or more robust error prevention measures such as atomicity?

Re data loss, I was referring to the possibility of unintended outline hiding causing headings to be misplaced. This regularly occurred to me when I was trying to manage large single-file Org outlines, which is why I developed Treefactor.

There is indeed an idiosyncratic workflow, which I will publish as a replacement for GTD. Treefactor is the subset of functions I thought suitable for a general audience.

I think using something other than level-4 headings for mass transit greatly increases the probability of user error and grief, but I can add a customization option to permit that choice easily enough.

I assume that part of your difficulty figuring out what Treesort does is the non-idiomatic elisp, so I'll defer anything else until I've implemented a step on my release checklist to delete the extraneous comments structure and whitespace.

One other question, related to the "Treefactor" name. Technically, restructuring prose is not parallel with refactoring code. To correct this mismatch, I can include a function that currently resides in my personal Spacemacs layer, which does assist prose refactoring: https://github.com/cyberthal/spacemacs-personal/blob/master/funcs.el

The function is for decomposing complex headings into single-thought headings suitable for filing. It's currently named trs-decomposing-mummy. It creates a copy of the target heading, then appends " | MUMMY" to the title of the original. In a new window, it creates an empty heading called "CANOPIC", after the canopic jars which store a mummy's organs. The user can then easily chop up the copy and moves the good bits to children of CANOPIC. Then delete the empty copy heading. Then liberate the children from CANOPIC and file them to their destined afterlife.

The original heading, now MUMMYfied, can be filed to the location that is least wrong. At some point it may be reexamined or resurrected. Eventually, once any possibility of its usefulness has expired, it can be safely deleted.

I understand that Org has an archive function, but this is a different use-case. Is this naming scheme too grim for MELPA? This one is quite memorable, doesn't collide, and the MUMMY headings shambling through one's files are impossible to miss.

[alphapapa]

The function is for decomposing complex headings into single-thought headings suitable for filing. It's currently named trs-decomposing-mummy. It creates a copy of the target heading, then appends " | MUMMY" to the title of the original. In a new window, it creates an empty heading called "CANOPIC", after the canopic jars which store a mummy's organs. The user can then easily chop up the copy and moves the good bits to children of CANOPIC. Then delete the empty copy heading. Then liberate the children from CANOPIC and file them to their destined afterlife.

I think, er, dissecting this explanation is indicative of what I perceive as, currently, the overall problem with this package: understanding what it does. That's a very creative explanation, but how will a random Emacs/Org user perceive it? It sounds bizarre and extremely idiosyncratic.

In contrast, this phrase is much more easily understood, and piques my curiosity, making me want to see it demonstrated:

decomposing complex headings into single-thought headings suitable for filing

This part does sound memorable:

This one is quite memorable, doesn't collide, and the MUMMY headings shambling through one's files are impossible to miss.

But, again, it's idiosyncratic. Users will have to wrap their heads around this seemingly bizarre idea. A few will get it, but most probably won't. So if you can generalize it a bit, use terms that are more expected in the context, users will probably understand it better.

I assume MELPA prefers the Outshine outline to be removed as well?

I don't know what you mean. Note that I am not MELPA, merely a fellow package author.

Since "Treesort" doesn't work, how about "Treefactor"? The goal is to streamline refactoring prose into a meta-outline composed of the directory hierarchy and text files containing outlines. One could merely use it for Dired filing, which is why I didn't present it as an Org package.

I think that would be a much better name. However, if its primary purpose is still Org-related, I'd still encourage you to put org in the name, otherwise users won't expect it to be Org-related, and they'll probably overlook it.

I use Treefactor to manage a daily intake of 1-10k words into a 600 mb git repo. That breaks down to maybe 10-100 headings daily.

That's the kind of explanation that ought to be at the top of the readme, preferably followed by a few simple demonstration GIFs. ;)

It doesn't delete user data. There are checks to prevent that. Are you suggesting writing unit tests? Or more robust error prevention measures such as atomicity?

I was suggesting both of those things. But my concern was mostly driven by the descriptions in the readme, which I may have misinterpreted. Then again, tests and error-checking are always a good thing. ;)

Curious, do you use org-refile, or do you move headings manually? org-refile handles some checking for you and is known to be correct, so it should generally be used when possible.

I think using something other than level-4 headings for mass transit greatly increases the probability of user error and grief, but I can add a customization option to permit that choice easily enough.

I would recommend being more flexible, yes. For example, in my Org files, the heading levels are meaningless. The hierarchy is flexible and arbitrary. To-do keywords and tags and properties are used for organization. I doubt that many users have a strict heading-level organization method. So if you want it to be generally useful, that will probably be necessary.

I assume that part of your difficulty figuring out what Treesort does is the non-idiomatic elisp, so I'll defer anything else until I've implemented a step on my release checklist to delete the extraneous comments structure and whitespace.

I didn't look at the code until after trying to parse the commentary and commenting on it. Then when I looked at the code, the hanging parens scratched my face, so I stopped looking at the code. ;)

One other question, related to the "Treefactor" name. Technically, restructuring prose is not parallel with refactoring code. To correct this mismatch, I can include a function that currently resides in my personal Spacemacs layer, which does assist prose refactoring:

This talk about "restructuring prose" and "refactoring prose" sounds very interesting. I think you should make those ideas prominent in the commentary.

[cyberthal]

Org-refile isn't suitable for my use-case. I think Org does a great job of handling metadata, and a poor job of handling bulk prose. Treefactor supplies that missing affordance. It lets Emacs refactor prose into single-thought headings and organize them into a directory-tree mindmap.

My org agenda search scope is restricted to the /1-Agenda directory, which contains almost no body text. Those headings link to UID org-links in the /2-Linked directory, where I store bulk prose. The latter has a deep constantly-evolving directory hierarchy.

Treefactor typically moves headings up or down one directory level at a time. This creates an emergent mindmap. Sometimes I'll file an Inbox.org with hundreds of headings this way, distributing them between several targets. If it's just one heading, then I skip multiple directory levels of course.

I considered renaming to "Treefactorg", but a triple-portmanteau is exponentially more awkward than a double. I doubt "org" without a dash would add much discoverability?

The Treefactor elisp prefix needs to be short so that commands can be invoked quickly, since the point of the package is speedy bulk filing. Some Treefactor commands are situational, not worth a keybinding, but would be a nuisance if they required typing a long prefix to access them in Helm. I'm willing to sacrifice prefix descriptiveness to obtain speed. "trs-" is unsuitable since Treesort is unsuitable, so I'll pick another free 3-letter prefix.

Another option would be to define a prefix keychord. I don't think that's justified. The main Treefactor commands need to be single keychords, for speed. The others are infrequent and batched, so can be Helm-searched once and repeated thereafter. The easy way to do that is M-x RET to repeat the last M-x Helm-searched command. repeat-complex-command also works, but its default keychord C-x M-: is too awkward.

Whatever prefix keychord Treefactor reserved would need to be an obscure and awkward one, since it has a weak case for one. And such keychords would be useless for batching the same command repeatedly, inbetween a few normal editing commands. Better to use the M-x trick.

Outshine outlines provide an outline structure to the code, with the following format:
;; * foo
;; ** bar
;; *** baz
The outline clarifies the relationship between various functions, but I can delete it if it's distracting.

I found a package "lispy" that will make it easy to reformat the hanging parentheses.

Managing the Readme and package commentary has gotten awkward and will only get worse. I plan to publish a manual separately, as org-mode does. That should reduce the strain on the other two documents.

[alphapapa]

Org-refile isn't suitable for my use-case. I think Org does a great job of handling metadata, and a poor job of handling bulk prose. Treefactor supplies that missing affordance. It lets Emacs refactor prose into single-thought headings and organize them into a directory-tree mindmap.

My org agenda search scope is restricted to the /1-Agenda directory, which contains almost no body text. Those headings link to UID org-links in the /2-Linked directory, where I store bulk prose. The latter has a deep constantly-evolving directory hierarchy.

Treefactor typically moves headings up or down one directory level at a time. This creates an emergent mindmap. Sometimes I'll file an Inbox.org with hundreds of headings this way, distributing them between several targets. If it's just one heading, then I skip multiple directory levels of course.

I'll politely reiterate that that sounds very idiosyncratic. :) If you want other users to benefit from your package, I think you'll need succinct documentation and clear demonstrations.

I considered renaming to "Treefactorg", but a triple-portmanteau is exponentially more awkward than a double. I doubt "org" without a dash would add much discoverability?

That's a cute name. There are some Org-related packages without an org- prefix, so you might consider that. But I would encourage you to use org-treefactor, because it's descriptive, clearly Org-related, and discoverable.

The Treefactor elisp prefix needs to be short so that commands can be invoked quickly, since the point of the package is speedy bulk filing. Some Treefactor commands are situational, not worth a keybinding, but would be a nuisance if they required typing a long prefix to access them in Helm. I'm willing to sacrifice prefix descriptiveness to obtain speed. "trs-" is unsuitable since Treesort is unsuitable, so I'll pick another free 3-letter prefix.

That's not really necessary. You could easily provide a function that optionally defines short aliases to commonly used commands; that way the namespace wouldn't be non-prefixed by default, and users could still benefit from the convenience without having to manually define aliases.

Another option would be to define a prefix keychord. I don't think that's justified. The main Treefactor commands need to be single keychords, for speed.

That sounds ideal for a Hydra. Have you looked into that? You can have whatever keybindings you want, activating the prefix map with whatever key you want, and a nice menu showing the available commands. That would be very helpful for new users.

Whatever prefix keychord Treefactor reserved would need to be an obscure and awkward one, since it has a weak case for one. And such keychords would be useless for batching the same command repeatedly, inbetween a few normal editing commands. Better to use the M-x trick.

Generally you shouldn't bind a key to a prefix map yourself. You can recommend one in the documentation, but that should be left for users to do in their own configs.

Outshine outlines provide an outline structure to the code, with the following format:
;; * foo
;; ** bar
;; *** baz
The outline clarifies the relationship between various functions, but I can delete it if it's distracting.

I am quite familiar with Outshine. ;) I didn't know what you meant about it. By all means, use those headings; many packages do. I use (rx (1+ (syntax comment-delimiter)) (1+ blank) (1+ "*"))-style outlines in most source files, although in Elisp I use semicolon-only headings (3+ semicolons defines a heading, with more semicolons indicating a deeper level).

I found a package "lispy" that will make it easy to reformat the hanging parentheses.

Yep, Lispy is great.

Managing the Readme and package commentary has gotten awkward and will only get worse. I plan to publish a manual separately, as org-mode does. That should reduce the strain on the other two documents.

Good idea. I encourage you to focus on making your introductory documentation as clear and concise as possible from a new-user's perspective. Also, pictures are worth a thousand words, animations a million, etc. Some people are much more receptive when they're shown what something can do.

For generating e.g. info manuals, you may find Magit's manual instructive, as it's all generated from Org files. For a simpler example, you could look at org-super-agenda, which I based on Magit's method.

[riscy]

Without digging too deeply, I thought I'd provide the author here the results of a quick pass through the checklist:

M-x checkdoc (using version 0.6.1):

Warning (emacs): 
trs.el:174: All variables and subroutines might as well have a documentation string
Warning (emacs): 
trs.el:414: All variables and subroutines might as well have a documentation string
Warning (emacs): 
trs.el:418: All variables and subroutines might as well have a documentation string
Warning (emacs): 
trs.el:422: All variables and subroutines might as well have a documentation string
Warning (emacs): 
trs.el:428: All variables and subroutines might as well have a documentation string
Warning (emacs): 
trs.el:434: All variables and subroutines might as well have a documentation string
Warning (emacs): 
trs.el:440: All variables and subroutines might as well have a documentation string

M-x package-lint-current-buffer (using version 20190707.1456):

No issues found.

M-x byte-compile-file (using Emacs 26.2):

trs.el:370:1:Warning: Unused lexical variable `trs-text-object'

In trs-store-link-fold-drawer:
trs.el:535:8:Warning: org-store-link called with 2 arguments, but accepts only
    1

[cyberthal]

I'm working on a major version update. Variable 'trs-text-object' removed. org-store-link appears to accept two arguments, one optional.

The functions without docstrings are trivial. Should I add an empty docstring, state the obvious, or leave it as is?

The workflow is loosely based on http://brainstormsw.com , a program as old as Emacs that still sells copies today.

[riscy]

The functions without docstrings are trivial. Should I add an empty docstring, state the obvious, or leave it as is?

I hate to be pedantic, but all variables and subroutines might as well have a documentation string. There's a benefit, though -- users looking at the function through Emacs's help (apropos, describe-function, etc.) will get a clue from the docstrings that the functions are indeed trivial when they're trivial (or nontrivial when they're not).

[cyberthal]

No problem. That's a good point.

[riscy]

@cyberthal Ping! :)

[cyberthal]

I'm nearly ready to resubmit the package. However, I'll be traveling soon through October 5 and can't predict whether I'll be able to complete the last few release steps before or during that time.

I do have a draft of the new manual published. It has demo gifs and explains each function in detail.
https://treefactor-docs.nfshost.com/Dev/

I would like feedback on the naming conventions. E.g., should it be named as an org- package? I haven't implemented that yet, because last time it wasn't clear to reviewers what Treefactor does, which is relevant to the question.

If you would like to wait to review the manual until I've released the new version of the code, that's fine. Just posting my work in progress in response to the ping.

[riscy]

should it be named as an org- package?

This is fine, as long as it has its own namespace that is unique in MELPA and not confusing with any other packages out there. org-treefactor is a great package name in my books.

[cyberthal]

Ok. I'd prefer to name the package just treefactor-, so that I can distinguish the org-specific functions from the others by prefixing them like so: treefactor-org-foobar.

[riscy]

Yeah, that seems like a good name. :)

[riscy]

Hey, just checking in on this one -- let me know if the new version of the code is ready to go. :)

[cyberthal]

Short answer: I'm working on it.

Code's almost done. Need to integrate an Avy bug fix.
Setting up my git-annex network with Wasabi to back up the gif demos I made for the documentation.

[riscy]

Ping! :)

[cyberthal]

v2.0.0 documentation here: https://treefactor-docs.nfshost.com
v2.0.0 code coming soon.

[cyberthal]

v2 code released: https://github.com/cyberthal/treefactor

Looks like this merge request is obsolete now. Should I do something about that?

[riscy]

Looks like this merge request is obsolete now

You don't need to rebase or anything, if that's what you mean.

[cyberthal]

I mean that since the package name changed, the recipe should no longer be correct. Wondering whether I should fix that now. That might create a new thread though.

[riscy]

Oh, I see. You can update the package name on this branch, it should be fine. :)

[cyberthal]

I updated the recipe. Installation succeeds.

Flycheck gives two warnings on the Treefactor v2 package. One is a verb tense suggestion that is incorrect. The other is

#+begin_src flycheck
tro.el 96 1 error Symbol’s function definition is void: tro-alias-name-list (emacs-lisp)
#+end_src

Dunno what to do about it. I already defined that function. Aliases are working on my machine.

[riscy]

Ok! Taking a look. package-lint has recently been updated and is much more thorough, so there might be some surprises in here:

M-x checkdoc (using version 0.6.1):

Warning (emacs): 
tro.el:301: First line is not a complete sentence
Warning (emacs): 
tro.el:317: First line is not a complete sentence
Warning (emacs): 
tro.el:446: First sentence should end with punctuation
Warning (emacs): 
tro.el:614: Lisp symbol `avy-isearch' should appear in quotes

M-x package-lint-current-buffer (using version 20191103.1154):

17 issues found:

124:6: warning: Closing parens should not be wrapped onto new lines.
217:3: error: You should depend on (emacs "25.1") if you need `outline-hide-subtree'.
218:3: error: You should depend on (emacs "25.1") if you need `outline-show-children'.
219:3: error: You should depend on (emacs "25.1") if you need `outline-hide-body'.
238:3: error: You should depend on (emacs "25.1") if you need `outline-hide-subtree'.
421:5: error: You should depend on (emacs "24.4") if you need `dired-hide-details-mode'.
508:7: error: You should depend on (emacs "26.1") if you need `org-previous-visible-heading'.
527:64: error: You should depend on (emacs "25.1") if you need `vc-root-dir'.
566:20: error: You should depend on (emacs "26.1") if you need `org-previous-visible-heading'.
573:18: error: You should depend on (emacs "24.4") if you need `org-N-empty-lines-before-current'.
589:14: error: You should depend on (emacs "26.1") if you need `org-next-visible-heading'.
603:14: error: You should depend on (emacs "26.1") if you need `org-previous-visible-heading'.
636:20: error: You should depend on (emacs "24.4") if you need `org-N-empty-lines-before-current'.
638:11: error: You should depend on (emacs "24.4") if you need `org-N-empty-lines-before-current'.
661:11: error: You should depend on (emacs "26.1") if you need `org--line-empty-p'.
682:42: error: You should depend on (emacs "25.1") if you need `vc-root-dir'.
683:77: error: You should depend on (emacs "25.1") if you need `vc-root-dir'.

M-x byte-compile-file (using Emacs 26.3):

tro.el:96:1:Error: Symbol's function definition is void: tro-alias-name-list

Extra hints:

• tro.el#L147: It's safer to sharp-quote function names; use #'
• tro.el#L618: You can use not or null (but this expression looks like it could be rewritten for clarity)

I think the byte-compile error is happening because the first thing the compiler wants to do is expand your macros which results in evaluating expressions that involve tro-alias-name-list. Could you achieve the same with functions? I also discourage metaprogramming of this sort (I don't think it saves you much code and costs you a lot of complexity to define these 7 aliases).

[cyberthal]

I haven't finish the batch of changes from your last comment. In the meantime, answering this one:

It can be awkward for package authors when they're trying to get what used to be a personal tool ready for public consumption, but I think it's better if we can fix things like this when we see htem.

I included the alias option originally to comply with this thread's feedback. The idea was to permit a longer package name to be changed into a short prefix. I didn't wind up picking a long package name, so I guess there's no MELPA reason to have aliases. I do find the "leo-" aliases handy though.

If you want to change the default alias prefix to something other than "leo", what would that be? "Treefactor" is the obvious choice, but what would the point of a longer alias be? There needs to be a suggested value to prevent the user error of including a redundant"-".

I plan to create another package called "Textmind" with functions to support that workflow. For example, one Textmind principle is to enter the day's thoughts and activities into '1sprinting.org in chronological order, whenever feasible. The user manually notes the passage of time through the text by the following command:

#+begin_src
(defun leo-org-time-and-date-stamp-inactive ()
"Insert inactive timestamp of current time"

;; Calls org-time-stamp-inactive with universal prefix
(interactive)
(org-insert-time-stamp (current-time) t t)
)
#+end_src

One can use Treefactor without Textmind, but not Textmind without Treefactor. However, it would be annoying for a user to have to think about whether a function is part of Textmind or Treefactor. Even I would have to puzzle about that, if I were working on something else. So it's handy to alias them together.

Even after I release the Textmind package, I would still suggest "leo" as the common prefix between the two packages. It would be inaccurate to label Textmind functions with the "tro-" label, and annoying if a user wanted to view just the "tro-" functions for some reason.

[alphapapa]

@cyberthal I feel like we're expending a lot of effort trying to help you see your packages from the perspective of other users. "Treefactor," "Textmind," "leo", "tro"--none of those terms mean anything to anyone other than yourself. "Low Earth Orbit? Is this an Emacs package for Kerbal Space Program? Or for tracking satellites?" "Tro...lololol? Is this a package about humor or YouTube videos?" "Treefactor? Is this a package for computing the common factor between binary trees?" Do you see what I'm getting at?

You seem very concerned with symbol prefix length. This is not a concern. There exist packages in MELPA with much longer prefixes. That's fine, because they are descriptive and readable. Emacs has many completion tools to make typing and searching them easy.

One can use Treefactor without Textmind, but not Textmind without Treefactor. However, it would be annoying for a user to have to think about whether a function is part of Textmind or Treefactor. Even I would have to puzzle about that, if I were working on something else. So it's handy to alias them together.

I think that paragraph may indicate that you need to take some time to further refine your packages into cohesive, independent parts that are easily understood by users. As the Three Hard Things say, naming things is.

[cyberthal]

@riscy I implemented the changes. All checks pass. Over and out.

@alphapapa > Do you see what I'm getting at?

I'm open to suggestions for better names.

You seem very concerned with symbol prefix length. This is not a concern.

As a Spacemacs Helm user, I'm aware of features such as which-key and fuzzy completion that aid both discoverability and access speed. Symbol prefix length is a tradeoff. A long, descriptive package name is great for discoverability. On the other hand, it can encumber M-x access.

A discoverable prefix is good to have if the package is likely to sit installed but half-forgotten on the user's Emacs. That way he can accidentally be reminded of it. This won't help Treefactor, since the reasons to use it are abstract and counterintuitive to Emacs users. Users will either use it daily or won't install it.

A hydra or which-key completion menu is apt when it would be helpful for the user to spend some time perusing the menu and learning about the features of an expansive package. Org-mode is a great example of this.

A short prefix is good for quick fuzzy-completion-assisted access. Treefactor has two important user functions that are too infrequent to warrant a keybind: tro-org-refactor-heading and tro-org-dired-zinks. The latter is easily reached via M-x zinks. The short prefix is useful specifically for tro-org-refactor-heading. I access it with M-x tro- refa RET

Inductive incremental refiling and refactoring text can absorb one's full concentration, depending on the subject matter. It is normally the hardest part of my work. Brain burnout from this task is my productivity bottleneck. Therefore Treefactor command access should be as thoughtless and automatic as possible - pure muscle memory. That's why I didn't bind tro-org-refactor-heading to a keybind. Groping for a half-forgotten keybind right before embarking on the hardest part of one's work is exactly the sort of mental friction Treefactor was designed to prevent. Instead I just remember the keyphrase "tro- refa". I'm reminded of the "refactor" keyword every time I see a refactored heading with "REFACTORED" appended to its title, so that sticks. That "refa" bit of automatic typing helps me remember that I'm inside a refactor operation if I get so immersed in the subject matter I lose track of everything else, which sometimes still happens.

The other important Treefactor user commands need a total of 4 keybinds. tro-refile and tro-refile-up need to be single-stroke keychords, for speed. They should be similar and adjacent keybinds, amounting to one mnemonic slot. tro-delete-this-buffer-and-file naturally wants the C-c C-k keybind; it's not my function or keybind originally. tro-org-store-link-fold-drawer just replaces the pre-existing C-c l org-store-link keybind. These are all frequent use and unforgettable.

It's an odd assortment of functions united more by the fact that they were all missing than a common category. Thus there's very little case for a hydra. Especially when the same thing can be easily accomplished by M-x tro-, for anyone seeking a summary. The keybind to access the hydra would be harder to remember than M-x tro- refa.

This is a lengthy reply, which reflects that I have put a good deal of thought into the division of my packages.

[alphapapa]

I'm open to suggestions for better names.

I already made some suggestions back in August. Since then you seem to have moved from treefactor, which is at least words, to tro, which is neither a word nor an abbreviation for words (i.e. tro does not seem like an abbreviation for treefactor).

This is a lengthy reply, which reflects that I have put a good deal of thought into the division of my packages.

It is indeed, and indeed you have--but that thought seems to be applied from the perspective of your personal workflow and jargon, not from the perspective of average Emacs/Org users. I can imagine that many of those users could benefit from your tools--but first they have to have a handle on them, a name that is memorable or at least descriptive. tro is neither. Neither is leo (which sounds like the Leo text editor, or Low Earth Orbit, neither of which is relevant here, and certainly nothing like "Treefactor").

Do you see how this is hard for users to grok? I'm not a genius, but I'm an experienced Emacs/Org user and package author, and I'm having a hard time grokking this myself. Imagine all the other Emacs/Org users trying to get a handle on it. We're trying to help you see from users' perspectives so your tools will have the best chance at being discovered and appreciated.

[cyberthal]

I already made some suggestions back in August.

I recall you wanted an org- prefix, which I declined in order to have a secondary org- prefix in front of the specifically Org functions. I am asking about names for the package proper, since you don't like Treefactor.

tro is a phonetic misspelling of "throw", in the sense of throwing things into a box, which is the analogy used in the documentation for inductive iterative refiling. It's the best mnemonic handle this collection of concepts has, I think.

It is indeed, and indeed you have--but that thought seems to be applied from the perspective of your personal workflow and jargon, not from the perspective of average Emacs/Org users.

I would be happy to put it in common parlance, but there doesn't seem to be an existing one that will serve. Do you know of a better way to express "inductive iterative refiling", for example?

I don't think this package is worth installing unless one at least somewhat adopts the philosophy behind it. It's not very impressive technically, nor useful practically, unless one uses it to save a huge number of repetitive actions due to a workflow that differs from Org's and Dired's current philosophy.

I've already stripped out everything idiosyncratic that I can and still leave its basic function intact. Remove anything else and it won't be able to do fast inductive iterative refiling.

I'm hoping that this is more difficult for Emacs experts to grasp because they already have an established and different workflow. Lots of people from the Windows world were able to grasp BrainStorm, and the fan group was by no means restricted to the cognitive elite. I'd say the crowd was similar to fans of Tony Buzan's mindmapping, but less artsy and more techy.

[alphapapa]

I already made some suggestions back in August.

I recall you wanted an org- prefix, which I declined in order to have a secondary org- prefix in front of the specifically Org functions. I am asking about names for the package proper, since you don't like Treefactor.

I don't necessarily object to Treefactor--not that my personal preferences are relevant, anyway. Treefactor seems like a more descriptive name (i.e. it's about refactoring trees), so I would say that it's a much better name than tro, trs, leo, etc.

tro is a phonetic misspelling of "throw", in the sense of throwing things into a box, which is the analogy used in the documentation for inductive iterative refiling. It's the best mnemonic handle this collection of concepts has, I think.

Well, that's kind of cute, I admit. If you use that, I think the first sentence of the documentation (and package description) should explain what "troing" is. But I'd still encourage you to use something like treefactor instead.

It is indeed, and indeed you have--but that thought seems to be applied from the perspective of your personal workflow and jargon, not from the perspective of average Emacs/Org users.

I would be happy to put it in common parlance, but there doesn't seem to be an existing one that will serve. Do you know of a better way to express "inductive iterative refiling", for example?

I don't think anyone knows what "inductive iterative refiling" is, except you. ;) I would suggest considering a less jargony initial description, even if it were more verbose, e.g. something like, "Organize complex information by gradually refining it into distinct hierarchies" (assuming that's actually a valid description of what "inductive iterative refiling" is).

I've already stripped out everything idiosyncratic that I can and still leave its basic function intact. Remove anything else and it won't be able to do fast inductive iterative refiling.

I don't mean to tell you how the code or functionality should be organized, just that it would benefit from being described in a way that users can more easily understand.

I'm hoping that this is more difficult for Emacs experts to grasp because they already have an established and different workflow. Lots of people from the Windows world were able to grasp BrainStorm, and the fan group was by no means restricted to the cognitive elite. I'd say the crowd was similar to fans of Tony Buzan's mindmapping, but less artsy and more techy.

I'm not familiar with those references. Most Emacs/Org users probably aren't, either. :)

[alphapapa]

By the way, I looked at your new documentation. You've clearly put a lot of work into it!

One point I'd raise about tro. You write in the manual:

So why not use the term “throw” instead of “refile”, if it fits better?

Because everybody already knows that “refile” means moving text and files.

By contrast, the word “throw” has a pre-existing conflicting Emacs meaning - “throw” is a built-in function in Emacs’ C source code. In general “throw” is coder jargon for an unrelated concept.

So we can’t use the term “throw”, even though it’s intuitive for non-coders. However, keeping an echo around in the phonetic misspelling “tro” can’t hurt. If anything, the misspelling is a reminder of just how simple and dumb Treefactor’s basic operation is. That’s not a bad thing. Even supercomputers are built out of 1s and 0s.

I disagree: there's nothing wrong with using throw in this context. A command with throw in its name wouldn't necessarily be related to catch/throw in Elisp, especially since most users won't also be Elisp programmers. There are other terms in Emacsland that have multiple meanings, e.g. "killing text" means putting it on the kill ring, where it still "lives," while "killing a buffer" means destroying the buffer and all the data related to it. So I think you could safely use "throw" instead of "tro", and I think that would be much more descriptive for users.

[cyberthal]

"Organize complex information by gradually refining it into distinct hierarchies" sounds good.

Maybe I should emphasize that Treefactor is for mindmapping. It adds some commands to Dired and Org to quickly move many small chunks of information around in small hops. This lets you gradually build a mindmap that matches how you think - even if don't know what the big picture is.

A mindmap graphic would be a nice opener:
https://ccsearch.creativecommons.org/photos/84a609ea-3ce4-42a5-b732-8cadc5a81053

"Treefactor" stands for tree refactoring. The tree's branches are directories and files, and the leaves are outlines within files. "Refactoring" means moving information around without changing the contents.

"Treefactor" relates to tro- via TReefactOr.

I disagree: there's nothing wrong with using throw in this context. A command with throw in its name wouldn't necessarily be related to catch/throw in Elisp, especially since most users won't also be Elisp programmers. There are other terms in Emacsland that have multiple meanings, e.g. "killing text" means putting it on the kill ring, where it still "lives," while "killing a buffer" means destroying the buffer and all the data related to it. So I think you could safely use "throw" instead of "tro", and I think that would be much more descriptive for users.

Still, I'd rather not trigger the "throw" objection in every programmer the first time he encounters the package. "Treefactor" has 676 Google results, and no technical ones on the first page. They're all arboreal. Even the programmer's misinterpretation at least gets the keywords right - tree and refactor.

Maybe I should start spelling it TReefactOr though, at least in the documentation, to ham-fistedly emphasize the connection with tro-. As a bonus, that would combine all three key concepts into the name - refactoring trees via throws.

So, the unique selling proposition is, "Restructure your messy Org documents,"
and the opening sentence is, "TReefactOr refactors trees via throws."

To extend the metaphor, the objects are leafcutter ants marching in lines towards their destination. Ants have no clue where they're going, they just follow the trail. Sometimes they even get confused and circle for a bit, but it all works out in the end.

https://ccsearch.creativecommons.org/photos/e99cfc67-d18e-46ca-8186-9c0408d8c800

Also, treefactor as prefix has M-x fuzzy completion collisions with other instances of "tree" and "refactor", whereas "tro-" is unique.

[alphapapa]

"Treefactor" relates to tro- via TReefactOr.

Yes, but the "O" is not the beginning of a word within it, so it's very awkward to use it as part of the abbreviation.

Still, I'd rather not trigger the "throw" objection in every programmer the first time he encounters the package.

throw and catch are actually very rarely used in Elisp code. I really don't think anyone would object.

Maybe I should start spelling it TReefactOr though, at least in the documentation, to ham-fistedly emphasize the connection with tro-. As a bonus, that would combine all three key concepts into the name - refactoring trees via throws.

That's not quite what I--

So, the unique selling proposition is, "Restructure your messy Org documents,"
and the opening sentence is, "TReefactOr refactors trees via throws."

N-no, that's not what--

To extend the metaphor, the objects are leafcutter ants marching in lines towards their destination.

...Make it so.

[cyberthal]

When I talk about my personal pain points with Treefactor, it's idiosyncratic - not because other early adopters won't have them, but because they'll have that and a whole lot more. Assuming the subject matter is complex and important, inductive iterative refiling is unavoidably hard. Otherwise the user wouldn't need to think about the subject matter.

This is especially true when one attempts to scale up to a full-mind mindmap. Maintaining a partial mindmap is its own exercise in boundary frustration. I've spent years growing accustomed to full-mindmapping and can no longer appreciate a newbie's pain, except by looking at the appalling state of my pre-Textmind records - which is painful enough.

I can't simplify the Emacs environment in general, but at least I can make Treefactor distraction-proof. The user will have enough trouble reconciling all the contradictions he never knew existed in his thought. The more fluid Treefactor is, the less likely he will sit and stew instead of moving things around and breaking the deadlock. Sometimes it's necessary to refactor incorrectly repeatedly until the chaos uncovers the right answer - just like breaking something to figure out how it works. The longer he can maintain his trance while refactoring without UI-nuisance interruption, the faster he'll find the answer. Immersed attention span has an exponential impact on solution quality and speed, not linear. That's why I care much more about UI than discovery. Treefactor must spread by buzz anyway - nobody knows he needs this.

If it were just me not wanting to learn an extra keybind and preferring to type for M-x completion instead, then I wouldn't hesitate to pick the treefactor- prefix. But I'm confident many users will be much worse about this than I am.

Another argument against a Treefactor hydra - it's more user-error prone than M-x or distinct keybinds. Just a single typo or misreading at the final hydra selection can trigger the wrong function, and it's hard to tell what you did wrong. I would know how to recognize what happened, but a newbie who didn't RTFM wouldn't. Which would make undoing the action much more difficult. Meanwhile, experienced Treefactor users shouldn't use a hydra because it's too slow.

[alphapapa]

Another argument against a Treefactor hydra - it's more user-error prone than M-x or distinct keybinds. Just a single typo or misreading at the final hydra selection can trigger the wrong function, and it's hard to tell what you did wrong. I would know how to recognize what happened, but a newbie who didn't RTFM wouldn't. Which would make undoing the action much more difficult.

A hydra would probably be a great way to help new users get their feet wet and learn what commands are available.

Anyway, regarding undo, it would probably be a good idea to implement appropriate commands with the macro org-with-remote-undo from org-macs.el.

[cyberthal]

Ok, I can put it in, but I'll recommend against using it in production.
I'll have to check out the undo thing.

[riscy]

I've experimented with using tro a little bit and things work the way I would expect them to. I had another last scan of the code and I'll leave a few comments at the end of this.

I think the above was a pretty good discussion. @cyberthal, I think you are you are trying in good faith to make sure that your users will have a smooth UX by being very particular about details. It may be worth bearing in mind that Emacsers tend to be pretty good at choosing and defining aliases and keybindings when they personally need them, and can recoil at having these decisions made for them, so it is possible to err on the side of too much. I hope you can see the point there.

By the same token, I don't think a hydra needs to be included (this would add another dependency anyway, and it sounds like you're not excited about it in the same way that me and @alphapapa aren't excited about the "leo" namespacing). You could document how one might create a hydra if desired, but I'd leave that up the individual users to add if they want.

Some last comments on code:

• Please include "treefactor" in your package summary line (;;; tro.el --- Restructure your messy Org documents) -- if someone hears about treefactor or sees your repo, they might have trouble searching for it on MELPA.
• If I go to an org-buffer and call tro-refile but my next-window is not in dired mode and it is not somehow a valid outline, then I can get a "Below first heading" error.
• Check major modes with e.g.: (eq major-mode 'dired-mode), not with string-equal
• You're using both (select-window (previous-window)) and (other-window -1) within the same function -- do these do the same thing? Worth settling on one?
• progn inside cond isn't necessary (e.g. line 451, 552) and inverting the if on line 619 (i.e. (/= n 0)) will let you avoid the progn

[cyberthal]

Ok, I'll work on that. I've thought of a way to resolve the naming issue.

I'll rename the package "treefactor" instead of "tro". Then I'll give treefactor two opt-in aliases, the first prefilled with "tro" and the second blank. Later I can suggest in the documentation for the as-yet uncreated "Textmind" package to use "leo" as short shared prefix that joins the two packages.

I'll rename "tro-refile" to "treefactor-throw", for vividness. Instead of "tro-refile-up", just "treefactor-up".

The M-x fuzzy completion collision I mentioned between "treefactor" and "refactor" doesn't matter. "ref" isn't unique anyway, and typing "-refa" is beneficial. So no UX is lost.

My goal is to attract nontechnical non-Emacs users to Spacemacs for GTD and mindmapping via Treefactor, so I need a easy set of recommendations for that purpose. I don't expect users with existing setups to follow.

It'll take me some time to rewrite everything. "tro" doesn't need to be woven into the documentation anymore, as a major concept.

[cyberthal]

I've done riscy's suggested code edits except:

You're using both (select-window (previous-window)) and (other-window -1) within the same function -- do these do the same thing? Worth settling on one?

They don't do exactly the same thing. other-window is an interactive function. I use it for the last window selection step, because it triggers golden-ratio-mode to resize the windows appropriately. This doesn't need to happen while the function is working. Changing it would be undesirable.

All checks pass. Doing rename and docs rewrite next.

[cyberthal]

I finished the rename from tro to treefactor, and updated the documentation and recipe. I believe that's everything that was blocking acceptance. Still lots on the roadmap.

[cyberthal]

It turns out that adopting the name Treefactor did cause fuzzy-completion collision with my M-x refa habit, specifically with treefactor-up. Recovering from that error breaks flow, unfortunately.

I wouldn't reliably remember to use M-x -refa instead, while distracted by subject matter. However, a keybind mnemonic adjacency should work. clone-indirect-buffer-other-window should be bound to a single-chord keybind for serious Treefactor use, and treefactor-org-refactor-heading is simply a more robust version of that command. So the two keybinds should be adjacent: H-b and H-v respectively, occupying one mnemonic slot.

I'll update the documentation accordingly.

That collision is evidence there are too many "re*f" in the namespace. Changing treefactor-refile(-up) to treefactor-throw and treefactor-up helped. With the recommended keybinds, the remaining M-x collisions don't matter. If users ignore my recommendations, I assume they know what they're doing. They can always get a shortlist with M-x tro-

[riscy]

If users ignore my recommendations, I assume they know what they're doing.

Agreed - and this is what every package maintainer needs to do.

tro to treefactor

Your elisp file is called "Treefactor.el" now, but elisp files are downcased in virtually all situations, especially when the package name is treefactor and your functions are prefixed with treefactor-. It's sort of the community style and may cause issues down the line -- would you be willing to downcase the filename?

[cyberthal]

elisp files are downcased

My mistake; fixed.

[riscy]

Great. Thank you for your patience with this -- I realize the process was a long one.

Welcome to MELPA! :)

[cyberthal]

I should thank ya'll for your patience. The elisp ecosystem requires expert oversight, and I'm a beginner. I'll link this thread in the documentation credits.

Lots to do to popularize Treefactor. After looking into ways to produce documentation in org-mode, I picked Hugo, which supports Org and has Emacs packages. So I can Treefactor the Treefactor docs.

[riscy]

Treefactor the Treefactor docs

💥