Repository navigation
Replies: 9 comments 13 replies
|
Was there a design discussion/plan before you started that endeavour, that I can follow a bit on the choices made? Since it is huge work, not humanly reviewable by one person whilst keeping the quality level we are known for, pushing some independent, unrelated fixes by themselves in PRs is a good idea whilst waiting for how to proceed for the big changes. I'm not a usual user of temporal tools, and don't understand enough of what it does to know if it did the right thing or not. |
|
Glad to meet you "in person" (I've seen you a lot in code:)! |
|
Hi Sören, fully agree with Anna and Niklas, great to see you back here! I do use the temporal framework on a daily bases for operational services. So I am particularly interested in that part of the code, and like Anna I would happy to get your improvements in. I browsed through your suggested changes (the markdown files and peeked in some of the code). It is quite a bit of stuff. There may be some architectural consideration (like the messenger, parallelization in the light of Python 3.14, time zone handling ...). But also, what is the most user- and maintainerfriendly approach for the future? Can it be preferable to completely replace some userfacing functions rather than amending them, for example... As for the integration of the changes, I am available for PR reviews. And I agree with what the others said here before, if you can split it up in smaller chunks like you already suggest it will be more doable to review. Though I also think it will take some time to get through all of it. As the saing goes: How do you eat an elefant? One bite at a time... Like Anna I would be interested in understanding a bit more about the benefits of the changes (apart from documentation and bug fixes). I noticed you highlighted some preformance / efficiency improvements. Maybe a PR against main in your fork could help understand a bit more? Anyway, looking forward to working with you on getting this in. Cheers |
|
Thank you for all the nice comments. The changes are to make the code better maintainable, not to make it faster. My original implementation was really crappy, especially the temporal algebra, and i wanted to split the large classes into smaller ones and re-organize/rewrite it. I also wanted write much better documentation with shell and python example. I solved some issues and two pull requests, but i carefully avoided to interfere with running PR. So the benefit is a working temporal algebra, better code and better documentation. There are about 16000 tests now. This is a lot. I will try to reduce it or to make it faster, but i think we will not be under 15000 tests. Important things you must know: I am using AI to write all the code, tests and the documentation, it is much better than me and i think that the software problem is solved with the latest AI models. Its only a matter of token budget. However, if you have problems with AI generated code and documentation, no problem, i will keep it in my branch of grass gis. |
|
A practical question: Could you create a poc-PR against main in your own fork? Just so we can assess the impact a bit better. That would also give us an idea about CI runs with all the new tests and it would highlight changes. Another question is dependencies, the old version had dependencies on PLY which is unmaintained (#7897), and there was an optional dependency on dateutil (which does not seem to be needed any more with Python >=3.11). Does the new implementation remove those dependencies? |
|
The AI-generated code is not a problem, the current challenge is how to review it and who has the ownership (in terms of code ownership - what @neteler calls code habitats). This relates to the human oversight being a limited resource as @echoix already said. In any case, I think it needs to go in piece by piece. A pre-AI example of a multi-phase work is my 2D raster mask improvement to support multiple masks at the same time similarly to how one can have more than one computational region. The main PR, PR #2392, is spread over 25 files for basically just reading env var instead of a hardcoded name, but it would be much worse if I didn't spread the work over 20+ PRs. More recently, I created a new parallelization-friendly, reproducibility-friendly, backward-compatible library for pseudo-random numbers. This grew out of what was conceptualized as a single PR, but I ended up creating tests and resolving command-line interfaces separately from the library changes and from the tool changes, so again splitting into smaller pieces to make it reviewable and merge with confidence. I take the same approach for other changes I did not share yet, and it seems to work for it promises, but it still does not remove the bottleneck of the human review. Nevertheless, smaller pieces should make the discussion about the technical choices more clear and focused. Statements like "this refactoring does not make it faster, but makes the code more clear" and "this change removes the need for X and Y so the code is faster for most use cases" are easier to evaluate each separately and for smaller changes. |
|
Reading the replies here, I've had another idea to add. If we are to use AI reviews of the PRs, since they were principally made with Claude models, wouldn't it be a good idea to have some Codex/Open Ai models or even like Gemini 4 Argon to take a look, hoping to avoid blind spots? Who has easy access to those models? |
|
We have had a zoom meeting and a discussion about how to proceed with the merge of my changes into grass. Thanks to everyone for the call. To summarize what we agreed on:
|
Uh oh!
There was an error while loading. Please reload this page.
Hi all,
over the last weeks I reworked the temporal framework in my fork and would
like to bring the work into GRASS. Before opening pull requests I would like
to agree with you on how to do it, because the change is large.
Branch (based on current OSGeo main):
https://github.com/holistech/grass/tree/temporal-framework
Overview of all user-visible changes compared to 8.5:
https://github.com/holistech/grass/blob/temporal-framework/temporal/temporalchanges.md
What changed, in short:
default for t.rast.algebra, t.rast3d.algebra, t.vect.algebra, t.select,
t.sample, the mapcalc tools and t.rast.aggregate(.ds). The classic
algebra stays selectable (-c or GRASS_TEMPORAL_ALGEBRA=classic) and the
old implementation stays in the code as a reference. Both are checked
against a recorded corpus of the old results; deviations are listed on
the changes page.
grass-addons use are kept and pinned by a contract test.
directories); the gunittest and shell tests are removed.
dataset and are tested.
r.univar/r3.univar, lib/raster3d (3D masks, precision of compressed
maps), the parser (check_int, check_double, the seed option), v.univar,
v.unpack, and the PostgreSQL backend of the temporal database.
Size: about 1,950 commits. In temporal/ and python/grass/temporal/ 880
files, +300,000/-60,000 lines, most of it tests and manuals; the library
code itself was rewritten at about the same size (+32,500/-32,300).
Outside of the temporal parts 211 files, +21,000/-1,300.
I did this work with Claude Code (AI agents, model Claude Opus 5.5; many
of the implementation and review agents ran on Claude Sonnet 5.5) under
my direction. I planned and reviewed each step, all changes are tested, and I take responsibility
for the code. I mention it as CONTRIBUTING.md asks.
My proposal:
temporal parts (r.contour, r.univar, lib/raster3d, parser, v.univar,
...), so they can be reviewed on their own.
manuals. Since the default algebra changes, an RFC may be the right
place to discuss it; I am happy to write one.
How would you like to proceed? Comments on the branch are welcome.
Best regards,
Sören Gebbert
All reactions