Tracking issue for 0.7 (new core + GTK4)

[diegogangl]

We're getting to the end of the 0.6 cycle, so it's a good time to start planning for 0.7 since we have lots of large, sweeping changes to land. This issue is meant to centralize, keep track of all the work and plan ahead so we don't end up with another dev hell scenario.

This is more a birds eye view of the big stuff we have to complete, the complete list of issues is in the milestone

Overall Status / Landing current PRs

• Land the new core PR New Core #668
• Pin an issue mentioning the drag & drop regression, to avoid duplicate bug reports — Fixed in GTK 4.6
• Start the Gtk4 port (Port to GTK4 #723) — superseeded by no.894 below
• Land Gtk4 + New Core + Listview Megaport #894
  • Please report issues in Diego's repo
  • Send merge requests to Diego's repo
• Review and fix the fallout from having merged all of the above 🡸 🌟 HELP WANTED 🌟
  See the remaining issues from the comments thread in MR #894 and:
  • Help fix and expand the test suite: Fix the test suite #1036
  • Please rebase your existing pull requests (if any) on top of the latest master,
    which contains the new revamped codebase
  • Report (or directly fix!) issues in the regular issues tracker here,
    with the label "Regression" (if they didn't happen in 0.6);
    if they are mission-critical we ought to target them to the 0.7 milestone.
  • Please send fresh pull requests for any known or new regression issue you'd like to help fix in general
  • Help fix the most essential plugins (see further below)

Polishing and completing the port

The new core was designed with Gtk3 in mind, with the idea to port over to listboxes and then onto Gtk4. @ranchester2 port changed the schedule a bit, so we have to move directly to Gtk4's listviews and models. We also need to finish some missing features.

• Add code for recurring tasks (@zeddo123 's territory). PR at New Repeating tasks #868
• Check backends code with @jaesivsm
• Port some properties in data classes to GObject properties (things that will need to be bound in widgets)
• Add a function to generate child models in store classes (using the children list)
• Add a model property to store classes
• Move the filtering code to several Gtk.Filter subclasses in a their own modules (probably in a models folder)
• Add a Gtk.Filter subclass for search
• Move the sorting code to several Gtk.Sorter subclasses in a their own modules too

Plugging in the new core

Slowly slide in the new core so we can validate how it works

• Initialize Datastore2 to Application and read the gtg_data.xml file
• Check the tasks are read correctly (using the dev console)
• Add a call to save with the new core in requester.save_datastore, set the path to gtg_data2.xml
• Check the xml files are the same
• Add calls to all the other parts of the requester , so both cores do the same (remove task, new task, etc.)
• Add calls to new core in editor to add, remove tags
• Add calls to new core in editor to add, remove subtasks
• Add calls to new core in editor to change status, "sync", etc
• Plug new core in calls in the main window

Porting to listviews

EDIT: I've done most of the porting to listview into these two sample scripts

Sidebar: https://gist.github.com/diegogangl/eba75d467854068ea75cc9a03ce61495
Tasklist: https://gist.github.com/diegogangl/3e53105bd4ac6e9953f926e3134dfb73

Basically everything works, except multiselection. We'll have to wait for Gtk 4.6 to solve that. The code is a lovecraftian horror but we'll clean it up when the time comes. It also includes content search (filtering) and sorting by added date, modifed date and tags.

• Design a UI for the new sorting, since we won't have treeview columns (kind of like the nautilus menu)
• Subclass Gtk.Box for each data type, so it can hold a pointer to the original object (not sure this is the best way though)
• Write the List item factory, DnD functions, etc
• Replace the treeviews in main window with a single listview, or 3 listviews (can we get away with just 1 with the pane system?)
• Figure out how to do the sidebar. Probably a Gtk.ListBox with rows for all tags/tasks with no tags, and 2 listviews for tags and saved searches mixed in the rows.
• Use the new filter based search
• Remove all calls and references to the old core
• Clean up whatever other mess we left behind

Revamping the plugin and backends API, and fixing plugins

As of 2021 this isn't so defined yet, but it's becoming clear that we need to change the old systems. Besides, all the previous porting and rewriting is going to break almost everything anyways.

• Defined whether backends should be something separate from plugins, or just a kind of plugin that runs a task regularly or at certain events (AKA syncing). In other words, removing backends and expanding the plugin api so "sync plugins" are possible
• Audit and update the plugin API for the new UI and core
• Port plugins and backends to the new stuff

2024 status: please help fix long-standing issues with some of our plugins, as they have been plagueing users for too long, sometimes preventing them from starting the app (as you can see with issues currently pinned to the top of this bug tracker).

• The most important of CalDAV-related issues
• The most important of Hamster-related issues

[diegogangl]

Made some mockups to sort out some ideas. I don't really know how Gtk4/libadwaita apps should look, but at least we can have a starting point:

SVG file if anyone wants to play with it:
https://mega.nz/file/Tx0iwCCZ#6qNAUNJuQwLQzxgYRif4h-kelLAtF_KyehbP8r1jvko

[zeddo123]

I'm not an expert in UI/UX, but I love it!
One thing though, apparently Glade won't provide Gtk4 widgets. here

[diegogangl]

Yup, we'll have to use Cambalache or write the XMLs by hand

[nekohayo]

Just to answer a question present in the mockup currently (since I'm not sure there's a ticket about that design specifically):

• I don't think it would look right if line heights vary, that's one of the things that annoys me quite a bit with Liferea's current "wide screen" layout mode, it just looks & feels broken/wrong.
• The ability to display 2 lines worth of content is interesting/nice-looking for those who don't have too many tasks, but if you have 500+ tasks, I don't think this will scale well especially since computer monitors are more height-constrained than width-constrained. It could make sense to display in two-lines mode when the current view (ex: selected tag) has few items in it, or when the screen is in portrait layout (as you'd get on a phone, or a portrait computer monitor) and auto-adjust to single line / compact mode otherwise when on a regular computer screen with lots of tasks...

[diegogangl]

Here's another take at the mockup, this time with single line rows. Maybe they could switch to 2 lines if the width is too small (though I don't know if this actually possible).

Let's see if I picked the right icons for start and due :)

[nekohayo]

Also, since you mentioned libadwaita, and your mockups kind of resemble its flat styling (but I only considered them to be wireframes more than high-fidelity mockups): if at all possible, please let's try to stick to pure GTK4 (and see how far we can go with that), and hold back from using libadwaita for the foreseeable future, if we can. I personally think libadwaita's "trendy" flat aesthetic is a regression in usability (I want my buttons to look like damned buttons that can be interacted with, not every widget component looking like a low-contrast solid-filled gray rounded rectangle!)

While I suspect I might be part of a minority, I surely am not alone to think that way (some people did complain about its buttons' usability issues, me included), and unless some drastic changes occur in the design team's orientation (which sounds really unlikely to me), I wouldn't really want to enforce that flat style onto our users unless we have a good technical reason to do so.

[nekohayo]

Well, disregard my personal "substance over style" IxD opinions on widgets usability for a minute, and consider my above argument from a multi-year strategic project management worldview: if we can achieve most/all that we want with pure GTK4, I would like to avoid rushing into adopting the extra dependency that is libadwaita; this may avoid painting ourselves into a corner technically, possibly save us some work/rework if anything happens to be short-lived in the stack we depend on (oh, how many times have I seen graphics-related libraries come and go!...), and possibly avoid drama around the whole styling thing; for users who would be unhappy with the app looking "not flat enough", I would think they would already set a flat libadwaita-like theme on their system that would apply to all their pure-GTK4 apps anyway, no?

That said, maybe fellow GTG contributors, upon technical evaluation, will overwhelmingly tell me that the functionality we need really requires libadwaita, at which point my argument would become moot, of course. I'm not sure we need all that to happen in the 0.7 cycle (especially if we want to get 0.7 out faster) unless some jaw-dropping UX-enhancing features in GTG are already made and strictly depend on that library (so far I can't imagine what they could be, however).

[diegogangl]

I just grabbed an SVG file I found on Gnome's gitlab :)
I'm a bit torn on libadwaita. It does enable adaptative UIs, automatic dark style and some other cool things. On the other hand, it apparently makes user theming harder (since it hardcodes the stylesheet), adds an extra dependency and I'm not entirely sold on the new stylesheet.

However, we could add a background to buttons. Timetrack does this:

In any case, there are no plans for/against libadwaita yet. The important thing in the mockups is the task list design and the sidebar

[thegranddesign]

From what I gather libadwaita is only a layer on top of GTK in a way to make GTK apps consistent in look and feel. I have to say that if the current look and feel of GNOME (eg files, calendar, etc) is in any way related to libadwaita then that should be the standard set for GTG. Incredible kudos to everyone working on this project but the current look and feel of GTG looks as if a developer designed it, which totally makes sense due to the history, but doesn't give GTG a "polished" feel.

@nekohayo I totally get what you're saying about buttons, etc but to be honest the web has already taught everyone that not only buttons are clickable. If I see that tag list with a count next to it in a sidebar, I know it's clickable regardless of what it looks like. If I want to be certain, I hover my mouse over and see if it has a hover effect.

At the end of the day, consistency with the rest of the apps being built for GNOME is significantly more important than doing what feels "right". If everyone did the thing that they "liked best" then we'd have apps that look completely disjointed. There's a reason why Apple has their style guide and everyone adhered to it. Everyone using their apps knew how their apps worked because they all looked the same. (And similarly with Google's "Material" style.)

I would also add that I think the architecture of the data should map to the UI first and foremost. I don't think we should design the data architecture and then back into the UI. I think we should design the UI and then determine the data architecture that makes those things possible (potentially future-proofing with some additional ideas). For example, do we really need infinitely recursable/nested tasks? Might it not be better to use a standard Project/Task architecture? Is the ability to nest infinitely conducive to "getting things done"?

I would also look at OmniFocus. Their UI is crap, but they do have a few concepts in their tasks and projects that are nice to haves (especially around recurrence).

@diegogangl I think your mockups look freaking amazing. 😍 One thing I would caution you on is the propensity to think of the data in mockups as "sacred". When designers do mockups they think in "perfect world" terms. I would challenge you to do:

1. a mockup that has 50 tasks and some of those tasks have > 10 levels of nested tasks (even though I think deep nesting is a bad idea, it's possible today and so needs to be considered)
2. put 50 tags in the tag list (some with extremely long names)
3. What does the "saved search" look like if it has a very long name?
4. How would a task count of 9,999,999 be handled?

Doing these "real world" but "imperfect" things will make a mockup able to be judged in a better light. I think the primary issue you'll find with the current mockup layout (similar to what @nekohayo pointed out) is that all of that padding is going to be a problem. I agree that there needs to be some breathing room between the items (versus current GTG's list view) but what you have there is (in my opinion) more than what will be desirable for actual use cases.

[thegranddesign]

Additionally I would point out that using libadwaita allows GTG to leverage changes made by the entire community rather than having to do all of that work itself. If the libadwaita community eventually decides that buttons styled differently work better for a certain reason then GTG doesn't have to re-implement that work. Again, I don't think anyone on this team is trained as a designer, so I think we should let the designers do what they do best and the developers can do what they do best.

[diegogangl]

Hey, sorry just saw your replies. Thanks for the thorough comments!

At the end of the day, consistency with the rest of the apps being built for GNOME is significantly more important than doing what feels "right". If everyone did the thing that they "liked best" then we'd have apps that look completely disjointed. There's a reason why Apple has their style guide and everyone adhered to it. Everyone using their apps knew how their apps worked because they all looked the same. (And similarly with Google's "Material" style.)

Good point

I would also add that I think the architecture of the data should map to the UI first and foremost. I don't think we should design the data architecture and then back into the UI. I think we should design the UI and then determine the data architecture that makes those things possible (potentially future-proofing with some additional ideas). For example, do we really need infinitely recursable/nested tasks? Might it not be better to use a standard Project/Task architecture? Is the ability to nest infinitely conducive to "getting things done"?

I'm sure @nekohayo can come up with examples :)
I think infinite nesting will work very nicely once we have a third pane to show the task contents in the main window. You can basically treat a task as a project. Subtasks can be project steps, or entire subprojects themselves. In other words, it would be flexible enough that you can scale from "walking the dog" to "build fission reactor" with the same UI.

@diegogangl I think your mockups look freaking amazing. 😍 One thing I would caution you on is the propensity to think of the data in mockups as "sacred". When designers do mockups they think in "perfect world" terms. I would challenge you to do:

Absolutely! I'm also the one writing the code, so I'm aware of how rough a sketch this is 😛

a mockup that has 50 tasks and some of those tasks have > 10 levels of nested tasks (even though I think deep nesting is a bad idea, it's possible today and so needs to be considered)
put 50 tags in the tag list (some with extremely long names)
What does the "saved search" look like if it has a very long name?

If Gtk allows we should probably lower the indent for subtasks/tags. Super deep nesting is definitely something to consider in the future, because I'm not sure what solutions are possible (extending the rows to add a horizontal scrollbar, hiding certain elements, etc). Long names would be ellipsized (I think Gtk already does automatically).

How would a task count of 9,999,999 be handled?

Like this -> 💣 💥

Doing these "real world" but "imperfect" things will make a mockup able to be judged in a better light. I think the primary issue you'll find with the current mockup layout (similar to what @nekohayo pointed out) is that all of that padding is going to be a problem. I agree that there needs to be some breathing room between the items (versus current GTG's list view) but what you have there is (in my opinion) more than what will be desirable for actual use cases.

I don't remember if that's from the original mockup or I went overboard myself. The new sidebar code so far has less padding.

Additionally I would point out that using libadwaita allows GTG to leverage changes made by the entire community rather than having to do all of that work itself. If the libadwaita community eventually decides that buttons styled differently work better for a certain reason then GTG doesn't have to re-implement that work. Again, I don't think anyone on this team is trained as a designer, so I think we should let the designers do what they do best and the developers can do what they do best.

I'm actually trained as a graphic designer and did web design for several years 😅 @nekohayo does branding, filming and witty writing. The main problem with GTG (whether UI or backend code) is that we are carrying a lot of legacy, and we're all swamped with work and other things, so things improve rather slowly and depend on massive rewrites (like the #894 PR) .

[kocio-pl]

Skipping the whole issue - infinite nesting is a basic feature for me, you never know how many nested layers is needed to get to the really actionable level.

[nekohayo]

Just selectively replying to a few of @thegranddesign's statements there, because I don't have much time nor energy lately:

I totally get what you're saying about buttons, etc but to be honest the web has already taught everyone that not only buttons are clickable. If I see that tag list with a count next to it in a sidebar, I know it's clickable regardless of what it looks like. If I want to be certain, I hover my mouse over and see if it has a hover effect.

You know this, and geeks and "digital natives" know this, but my 50-80-years-old family members don't. Some business owners I see using computers, also, are absolutely excruciating to watch. And even if you try to teach them these non-visually-obvious things, they keep forgetting and cursing whoever came up with these things, while you can only awkwardly stand, watch, and apologize to them for "the way things are". They reveal all the flaws of the modern style-over-substance UIs. I have thousands of UI design issues filed across GNOME and the freedesktop to prove my point.

I suspect you haven't spent enough years doing actual in-the-trenches usability studies and tech support to realize just how disconnected from the average non-techie layperson you can be. I did, and still do such observations these days; that's why I find the current incarnation concerning visually.

But my point is also that, just technologically+technically speaking, if we don't strictly need to depend on a specific feature of libadwaita beyond the standard GTK widgets, I don't see why we should go through that "just because", sounds to me like painting ourselves in a corner. It has to be a very clear win from a maintenance perspective vs the cost of potentially needing to rip things out some years down the road (if I had a dollar for everytime someone wanted to switch canvas/UI/style toolkits in some of the desktop projects I've been involved in...)

[...] I don't think anyone on this team is trained as a designer, so I think we should let the designers do what they do best and the developers can do what they do best.

Just to rectify the record here:

I actually am an interaction designer (I even have a hidden page about that), and usability-focused interaction design has been my primary way to contribute to FLOSS over the last ~20 years (along with QA and project management); I very seldomly develop code myself (I just know an abnormal amount of coding tricks to be able provide occasional guidance, or to know when something is/isn't trivial). In my day-to-day, my clientèle's needs are typically much higher-level than UI design, so I don't emphasize my design work much at all, because the value I provide in business management consulting completely dwarfs any application-level design work I do. Just because I have philosophical disagreements with the "flat" fast fashion doesn't make me clueless about modern design patterns ;) nor do I have final say in design matters here; at the end of the day, this is a community/patches-driven project, to which I just happen to provide opinions, guidance and support when time allows, hopefully preventing pitfalls projects commonly fall through.

[thegranddesign]

@nekohayo I'm just providing my 2 cents. I don't know Python so I can't contribute and am not doing to judge the final decisions made here. As someone who has a couple very popular libraries I know the burden of maintaining for free.

A couple points and I'll let this drop (unless @diegogangl has another amazing mockup to show 😍)

the "flat" fast fashion

I'm not exactly what you consider "fast". Flat has been around in major operating systems for over 10 years.

You know this, and geeks and "digital natives" know this, but my 50-80-years-old family members don't.

I don't see 50-80 year old non-techies adopting Gnome any time in the next couple decades

vs the cost of potentially needing to rip things out some years down the road (if I had a dollar for everytime someone wanted to switch canvas/UI/style toolkits in some of the desktop projects I've been involved in...)

And if I had a dollar for every time someone recreated an already existing abstraction because it "wasn't invented here"... 😉

Regarding infinite recursion, I still don't think it's necessary. I think trying to design a UI that is good for everything means it's best at nothing. Tasks are not projects.

Again, just my two cents. I'll be using GTG regardless because it's the best open source thing currently out there. ❤️

Keep up the great work everyone!

[nekohayo]

@diegogangl I don't know if this changes anything to your plans (or if that is already where you were headed), but in case it is somehow insightful or inspiring, the Nautilus maintainer blogged about his experience with migrating away from TreeView and to ColumnView, which apparently can also still do tree views (?!)...

[diegogangl]

@nekohayo ended up replying late, but yeah we can still do trees with Gtk.TreeListModel and Gtk.TreeExpander. We are already using those in the megaport. There's only one issue with hiding/showing the expander arrow dynamically. That's been solved (I think) in Gtk master

[gargullia]

@diegogangl Will the 0.7 version use Gtk4/libadwaita? I looked at your layouts and they look just like a dream! It's very similar to planner and gnome todo, but cleaner and more understandable.

The argument for using the libadwaita:

• is that it is developing rapidly
• they look beautiful (all in the same style)
• you can make an adaptive interface (for mobile devices)
• all new GTK4 functions will be supported
• the opportunity to participate in the GNOME Circle, to attract participants in the development, distribution of the application

[diegogangl]

@gargullia Thanks for the kind words. I'm leaning torwards libadwaita, but definitely not for 0.7. We're way over our heads with all the changes (you can check the #894 PR)

[gargullia]

@diegogangl You can also use Blueprint

Information from here: https://thisweek.gnome.org/posts/2022/09/twig-61/

Bottles was recently ported to Blueprint! Blueprint is a new markup language to create GTK interfaces. It is more efficient for development as the syntax is less verbose and more visually pleasing in contrast to XML. Thanks to its efficiency, Bottles’s codebase was reduced by 800 lines!

[nekohayo]

The big refactoring has finally landed! I have updated the description above, clarifying the next steps and providing some convenient links.

Anyone who may have been waiting to land their code: please rebase your branches to the latest master.
The new codebase is expected to be much better to work with.