Is there something like auto-init?

[Byron]

When using stg I regularly always bump into trying to stg new a patch on a new branch to be informed that the branch isn't initialized yet. This feels a little bit like a wrench thrown into an otherwise pleasant workflow.

Now I wonder if I have been missing something but couldn't find a flag in new to auto-initialize the branch as needed, nor could I figure out if there might be git-config options that stg would read by default.

If none of that exists yet, I feel that an --init option for new would do the trick, and shell aliases can take it from there.

What do you think?

---

A follow question to this one could be related to cleaning up the meta-data (that I know nothing about) that it initializes, but I didn't want to digress too much here.

[jpgrayson]

Currently, in order for a branch to be initialized with a StGit stack, either stg init must be called or the branch must have been created with stg branch --create or stg branch --clone.

The feature you ask about is something I've been thinking about for quite a while. My workflow is also often interrupted when I switch to a branch and try either stg new or stg uncommit and have it fail due to the StGit stack not being initialized. For StGit 2.0, I considered going as far as removing stg new entirely and making stack initialization implicit in all cases. Ultimately I did not do that because it seemed like a bridge too far considering the already massive scope of the rewrite.

Anyway, I am in favor of having a stgit.autoInit configuration value which causes the stack initialization to happen automatically for any stg command that attempts to create a patch. So new, uncommit, import, pick. And other stack interrogation commands would pretend there is an empty stack. This would include series, top, prev, next, and maybe a few others.

Having a complementary --init option to stg new, uncommit, pick, and import would also be good. It would help with discoverability of the stgit.autoInit configuration option.

---

The stack metadata that is initialized by stg init consists of:

• A tree (git object) containing the stack metadata which is disjoint from the repo's nominal tree structure. The main thing in this tree is a file (blob) called stack.json which contains, unsurprisingly, the state of the stack.
• A reference to the above tree refs/stacks/<branch>.
• Sometimes some local configuration in the branch.<branch>.stgit namespace.

You can inspect the stack metadata with a few git commands, for example:

• git show refs/stacks/main:stack.json
• git ls-tree -r refs/stacks/main

Deinitializing the StGit stack from a branch is done via stg branch --cleanup. This just removes the refs/stacks/<branch> ref and any local branch.<branch>.stgit configuration.

N.B. the reason that StGit keeps stack state in the repo's object database (instead of in plain files under .git which it used to do) is so that stack state history can be tracked. This enables the stg undo and stg redo commands.

I'm not sure that auto-deinitialization makes as much sense as auto-initialization. For example, emptying a stack, either via stg delete or stg commit -a, would not be a good signal for removing the stack state history. Not sure what other signal might be used to trigger auto-deinitialization.

[Byron]

I also noticed that when running stg init it might prompt for credentials if stgit.gpgsign or commit.gpgsign are turned on, which was unexpected as it only sets up its metadata which I consider a black box. Generally I was surprised to see it signs metadata commits even though I appreciate it signing user-facing commits.
Once auto-init is a thing this might be less of a problem as stg new should eventually also create a user-facing commit so it won't be obvious that it also signed metadata commits.

[jpgrayson]

The various configuration options are perhaps mostly covered in the man pages and online help, but spread out in a not so helpful way. It would be good if stg(1) covered all of them in one place. So that's a TODO.

Thanks a lot! That will be helpful indeed and is much appreciated.

I've documented the StGit configuration variables (ae884eb), so look for those in the next release. Thanks for motivating me to finally get that done.

[...] when running stg init it might prompt for credentials if stgit.gpgsign or commit.gpgsign are turned on [...]

I suspect most StGit users are like you (and me): they don't really want the stack metadata commits to be signed even if they do want the patches (regular commits) to be signed. So, making stgit.gpgsign independent of commit.gpgsign and defaulting it to false/off would probably be a better default. I'm thinking I'll make this change, but it will need to go into StGit 2.1, which will have some other breaking behavior changes, such as auto-init.

Speaking of auto init, I have a working proof of concept. But instead of having a stgit.autoinit configuration variable, I'm playing with the idea of just making auto-init the default/only behavior. My current thinking is that if a user is running a StGit command to create a patch, then lets do whatever needs to be done to create that patch, including the stack initialization, if necessary. Another way to say it is that I don't believe there are scenarios where a user runs stg new, it fails, and the user says "whew, glad that patch didn't get created on this uninitialized branch"; only scenarios where the users says "ugg, I haven't init-ed this branch yet--why doesn't the tool just do that for me?!".

The current question on the table is whether various other commands should be allowed to succeed as if there is an empty stack when the stack has not yet been initialized? E.g. should stg series succeed (but print nothing) when run on an uninitialized branch? And should stg push fail with "not initialized" or "no unapplied patches"? I'm finding the trade-offs for these harder to balance.

[Byron]

I loved everything I read here, thanks for making these improvements :)!

Something else in the back of my head is and in the light of auto-init is the accumulation of metadata. Is that even a thing or does metadata go away (or become unreachable and ready for gc) when the branch is deleted? Or is there any kind of cleanup that should be triggered from time to time?

E.g. should stg series succeed (but print nothing) when run on an uninitialized branch?

To my mind, now that initialization is an implementation detail, stg should probably never mention initialization. With that in mind, an uninitialized branch would always be one without patches. If that were a guiding principle, one would land at no unapplied patches for stg push naturally which also feels natural to me. One might be able to improve the error message to also state that the stack is empty in case the user forgot to create a patch in that particular case.

[jpgrayson]

I've pushed a change (d4255b5) that adds auto initialization. No new configuration variable. Patch creating commands (new, pick, import, and uncommit) auto initialize the stack, if it is not already initialized. Other commands that operate on the stack will do their business as if there was an empty stack. For commands such as series and show, that means the commands will succeed in a meaningful way. For other commands such as push, pop, top, etc, they will still fail, but the failure will be due to no patches in the stack rather than that the stack is uninitialized.

For commands that explicitly deal with stack history, such as log, undo, redo, reset, and repair, the behavior is unchanged because the stack metadata and its history is fundamental and thus cannot be hidden from users.

Something else in the back of my head is and in the light of auto-init is the accumulation of metadata. Is that even a thing or does metadata go away (or become unreachable and ready for gc) when the branch is deleted? Or is there any kind of cleanup that should be triggered from time to time?

Yes, stack metadata history does accumulate. Every stack modifying command will add a commit to the stack metadata ref (refs/stacks/<branch>). It does not naturally become unreachable and thus does not become a candidate for git's gc. The stg log --clear command exists because of this concern--it prunes the stack history while leaving the current stack state unchanged. Also, stg branch --delete deletes the branch along with its associated stack metadata reference. And stg branch --cleanup completely deinitializes the StGit stack while leaving the git branch in place.

To my mind, now that initialization is an implementation detail, stg should probably never mention initialization.

I don't think we can go that far.

With that in mind, an uninitialized branch would always be one without patches.

There are a bunch of use cases where a stack will transiently not have any patches, but where the history remains useful and valid. A trivial, but important case would be if someone ran stg delete .. and immediately regretted it and wanted to undo with stg undo. If the stack history was either truncated or deinitialized when the stack became empty, StGit's undo, redo, and log capabilities would no longer work.

If that were a guiding principle, one would land at no unapplied patches for stg push naturally which also feels natural to me. One might be able to improve the error message to also state that the stack is empty in case the user forgot to create a patch in that particular case.

The stg push case is covered by this change. Attempting to push on an empty stack will now result in a No applied patches error instead of a stack not initialized error.

---

The stg pull and stg rebase commands did not really fit the command categories I outlined above. Their behaviors remain the same (stack must be initialized for them to proceed), but I think reasonable behaviors could be defined for these commands when the stack is not initialized. So these commands remain on the table in my mind.

[Byron]

Thanks a lot for the heads-up and for putting in all this work! I will definitely be using this version from now on.

Also, I found the information on how to clean metadata very insightful. stg branch --cleanup is what I used thus far, and feel that I'd be using a command like stg gc if I could to remove everything that isn't referenced anymore due to the parent git branch being deleted or due to there being no patches present anymore. I am sure there is a lot of pitfalls with that which I am not seeing, but also think that it's too easy right now to have left-over branches in refs/stacks that don't have a matching git branch anymore and don't seem to be picked up by stg branch --list either. It's probably because I don't use stg branch --delete or any of its non-patch related features even though maybe that's expected.