Taskchampion status update

[djmitche]

I've been working on my add-rust branch for a long time now, and I've finally got it passing tests. I've pulled a number of other, separate PRs out of the branch and most of those are merged already. What remains:

• A bunch of [BREAKING] commits -- these remove existing functionality from Taskwarrior. The intent is to look at these individually and decide how urgently we need to re-add that functionality: before the rest of the branch lands? before the next release? eventually? never? Ideally I could get some help on some of these, or it's going to be another very looong time until they are fixed.
• Two commits to support reading tasks, and writing tasks, respectively. This is the "meat" of the branch, and both commits are pretty large. I'm not sure it's worth keeping them separate, to be honest.

I'd like to land this as a PR soon.

[smemsh]

could you enumerate the anticipated breaking changes?

[smemsh]

I use purge to truly remove the task, it remains in the database otherwise (it doesn't expire, maybe you're thinking of it moving to completed.data), this is useful to remove test tasks, accidents, etc so the data doesn't appear in searches etc.

as for changelog, losing task change history itself would be a big deal, I would delay TW upgrade rather than do that. is it only the UI linkage or the change history itself that would be lost?

[djmitche]

No, I'm thinking of expiration -- there is no more completed.data or anything like that :)

[smemsh]

Sure, the implementation details don't matter. As long as there remains a way to mark a task deleted without actually removing it (delete), and also a way to do an immediate, permanent delete of all traces (purge), there won't be a regression. Since delete and purge have decidedly different semantics, it makes sense to keep both commands intact...

Likewise, as long as all task changes continue to be logged, and upgrading Taskwarrior imports the old changelog, there won't be a surprise loss of information, or a regression in what data gets logged.

I hope you're also considering people that run with gc=off because they don't want ID numbers to change or tasks to be moved until they explicitly ask for it by running a manual GC.

[smemsh]

btw, I'm probably missing something, but if there's no separation of completed data anymore, and the new db backend is only used to store serialized records (no indexed field access), doesn't it mean all operations will slow down in proportion to the number of tasks (even completed ones)? Sure, it's just IO but then also deserialize and scan. Since people can have many thousands of tasks over time, this can eventually add perceptible latency. I think this was one of the reasons for completed.data in the first place, so pending.data could remain small and fast.

[djmitche]

It's a database, so there's still indexed access. The working-set is maintained as a mapping of working-set ID (small numbers) to UUIDs. There are a few places where things are slower than I'd like -- for example, generation of +CHILD and +PARENT tags requires scanning all pending tasks via the working set. Some of that can be cached, but it's still pretty slow. Still, there's ample room for optimizing those things once the integration is complete.

[djmitche]

I'll put this branch up for review once a few of the other PRs I have out get reviewed.

[djmitche]

I'll do this one breaking change at a time..

1. [BREAKING] Remove journal.info support #3060

[ryneeverett]

I'm not very familiar with taskwarrior's maintenance/governance structure. (I assume it's informal.) Are there specific stakeholder(s) who need to approve such changes? If so should they be assigned the pull request?

[djmitche]

It is pretty informal, and the project is quite slim on the "human resources" front right now. I've had conversations about this work with @tbabej that confirm the overall direction, although he may be too busy to approve specific PRs.

My approach at the moment is to land stuff when someone has had a second look -- and thank you @ryneeverett for doing so on several PRs in the last week or two! -- and assume that we can straighten out any bugs in the develop branch before shipping a release. I realize that's not ideal and wouldn't do that on a more active project, but it seems the best balance of review against progress in the project's current conditions.

[smemsh]

I'm confused by the responses, can you help me to understand the implications for the particular points?

• is gc=off going to be supported still (stable ID numbers until explicitly rotated)?
• will delete function (mark as deleted, but still exists) still be supported?
• will purge function (remove all traces) still be supported?
• is history going to be LOST when upgrading or simply not exposed in the UI until that's implemented?

Removing any of these capabilities would be showstopper regressions for me to upgrade, personally...

[masaeedu]

@djmitche I would also like to be able to see edits beyond just start and stop times, e.g. changes to task descriptions, additions/removals of tags, moves between projects, etc. I realize that the broader requirements make things more difficult to implement, but this is why I was interested in seeing if I can help alleviate some of the pain involved.

[djmitche]

Got it, thanks! I do find it a useful feature!

[smemsh]

Re: Purge: I use it to remove junk tasks, which were added for testing/whatever or bad entries I want to do over again without dirtying history. I have a bunch of wrappers and hooks and sometimes have bogons created which I might not notice until later. These need to be removable. Sure I could mark them all with something and implement ignores in everything that wraps them, but actual deletion (as though they never existed) seems like a basic feature that solves a lot of cases at once.

Re: History: I use history quite a lot. Knowing exactly when I made changes to things, what I did, how work was done, how status changed over time, tags that got added and removed and when, is very valuable to understanding issue progression. Losing this would be a big regression to my workflow. Especially all the lost history not being preserved.

I will avoid upgrading until this stuff is implemented. Taskwarrior has been frozen for a long time really anyways, so that state will continue, at least for me. Eventually these things will hopefully get implemented...

I do think the larger effort is important and looks very promising, and without being able to contribute anything at the current time I can't really say much. Code is king, so... it is what it is.

[djmitche]

Thanks for the details. Does just task delete not work for getting bogus entries to go away?

[smemsh]

Deleted tasks are still in the database and show up in various listings, until purged. I will now have to exclude deleted tasks in a lot of places and for impromptu commands. And some things still don't accept virtual tags like -DELETED. For example they cannot be used with 'tags:' filter, which makes it more difficult for libraries. I suppose I can reject based on 'status' but it's a lot of places things have to be changed, where permanent removal is just better.

I'd almost suggest changing the definition of delete to be that of purge since that's usually the intention, but some people probably want to retain deleted tasks. Different people do use taskwarrior differently. Note that 'purge' was added just in 2.6.0 and probably for a reason. The commit is from @tbabej and doesn't say why it was added, and issues/PRs don't seem to mention it much, so I'm not sure, but I have occasion to use it with some frequency... usually any time a task is deleted, although I don't want it to rotate my ID numbers until I decide on it, so I run with gc=off and use this:

taskgc () 
{ 
    task rc.gc=on rc.verbose=nothing // list > /dev/null;
    (($(task +DELETED _unique uuid | wc -l))) && task rc.bulk=0 +DELETED purge
}

to make the action explicit.

A global rc setting to never display deleted tasks in reports/filters and never match them might work.