Sub-Events for Gramps — Design Plan

[dsblank]

This is a proposal for implementing the idea of super- and sub- events. I've tried to capture many of the ideas of the following conversations:

1. https://gramps-project.org/bugs/view.php?id=10672 - issue ticket
2. https://gramps.discourse.group/t/whats-in-development-for-gramps-6-2/9641/9?u=dsblank
3. https://gramps.discourse.group/t/whats-in-development-for-gramps-6-2/9641/14?u=dsblank

and others as well. If we can come to consensus on this, we can begin development.

Overview

Items marked "deferred" will be addressed in Phase 5.

Problem Statement

Gramps currently models events as a flat list. Every event — whether a single
birth, a multi-year military campaign, or an entire national census — sits at
the same level. There is no way to express that one event is a component of a
larger event, or that a set of events forms a meaningful group.

This creates real research problems:

• A researcher tracking an emigration journey must create dozens of independent
  events with no structural link between them. Finding all the legs of one
  journey requires manual searching or complex filters.
• Census projects spanning states, counties, and households cannot be
  represented as a hierarchy — everything ends up in one undifferentiated list.
• Military campaigns, ship voyages, and migration routes all have natural
  internal structure that Gramps currently cannot capture.

The feature request from the community asks for sub-events: the ability to
attach an event to one or more "super-events", so that events can be browsed
and understood as a hierarchy rather than a flat list.

Motivating Example

A researcher documents an 1882 emigration from Rinteln, Germany to Prineville,
Oregon. The journey takes 84 days and involves many discrete steps:

Emigration 1882  (main event, linked to the family)
  ├── Leave home — Rinteln
  ├── Walk to Bremen with cart (10 days)
  ├── Arrive Bremen Harbour
  ├── Register as passenger on D/S Montebello
  ├── Montebello Journey, Bremen → NYC
  │     ├── Departure from Bremen
  │     ├── Bad weather (5 days)
  │     ├── Little Katarina falls sick (day 6)
  │     ├── Little Katarina dies (day 8)
  │     ├── Emergency port call, Hull (repair — discovered later)
  │     └── Arrive New York Harbour
  ├── Register at Ellis Island
  ├── 5 days in New York City
  └── Travel to Oregon
        ├── Train
        ├── Wagons
        └── Arrive Prineville, Oregon (day 84)

Later, the researcher finds two other families who also sailed on the
Montebello from Bremen with Oregon as their destination. Their journey took 103
days — 20 more than expected. By examining all events linked to the place "D/S
Montebello", the researcher discovers the unplanned stop in Hull. That stop is
registered as a sub-event of "Montebello Journey 1882", linked to the place
Hull, England — and because that voyage event is already linked to the place
"D/S Montebello", the Hull stop appears naturally under both the Emigration
hierarchy and under a separate "Ship Journeys" hierarchy the researcher has
also built.

This is the capability the feature must support.

---

Terminology

The words parent and child are deliberately avoided. In a genealogy
application those words refer to people, and using them for events would create
confusion in both UI labels and in conversation.

The community discussions naturally use "main event" and "sub-event".
We adopt these terms for user-facing labels and introduce "super-event" as
the precise technical counterpart to "sub-event" (it simply means "above in the
hierarchy" with no genealogical connotation).

Context	Term
User-visible label (containing event)	"Main event" / "Part of"
User-visible label (contained event)	"Sub-event"
Code field name	super_event_list
Code concept	super-event / sub-event

---

Key Design Decisions

1. List of super-events, not a single handle

An event can belong to more than one super-event hierarchy at the same
time. In the emigration example, the Montebello voyage event belongs to both
the "Emigration 1882" hierarchy and an independently maintained "Ship Journeys"
hierarchy. A single optional handle cannot represent this.

The field is therefore super_event_list: list[EventHandle] — a list of zero
or more handles to super-events.

This mirrors the Place model exactly. Places already support multiple
parent places through placeref_list, allowing a village to appear in more
than one administrative hierarchy (civil parish and ecclesiastical parish, for
example). Sub-events follow the same pattern.

2. Plain handle list, not a wrapper object

Gramps' PlaceRef secondary object wraps a handle plus a validity date because
a place has no date of its own. A village might belong to one county in 1800
and a different county after a border change in 1850, so the relationship between
places needs its own date to capture that.

Events already have a date. A sub-event's own date already says when it
occurred; the relationship between a sub-event and its super-event carries no
additional temporal information that is not already on the event itself.

Using a plain list[EventHandle] keeps the implementation simple and avoids
introducing a new secondary-object class.

3. Reference stored on the sub-event, not the super-event

Each sub-event stores the handles of its super-events. The super-event does
not store a list of its sub-events. This is the same direction as placeref_list
(each Place stores its parent Places, not the other way around).

Pros:

• Deletion of a super-event is handled cleanly by the existing handle-reference
  machinery — the sub-event's list is updated automatically.
• No referential integrity problem: a super-event can be deleted without
  leaving orphaned forward-references.
• Querying "what are the sub-events of X?" requires scanning all events, which
  is acceptable at current Gramps database sizes. An index can be added later
  if performance becomes a concern.

Cons:

• A query for sub-events of a given event is O(n) in the number of events.
  For very large databases (>100 000 events) this may be slow. Mitigation: add
  a secondary index in the database backend at the same time, or defer until
  needed.

4. DAG, not a strict tree

Because an event can have multiple super-events, the structure is a
directed acyclic graph (DAG), not a simple tree. The hierarchy view must
handle events that appear under more than one node. Cycles (an event being its
own ancestor) are forbidden and must be checked at save time.

---

Implementation Plan

Phase 1 — Data Model

gramps/gen/lib/event.py

• Add super_event_list: list[EventHandle] field, initialized to [].
• Update serialize() — append the new field as the last element of the tuple
  (keeps all existing serialized data valid).
• Update unserialize() — if the field is absent (data from an older Gramps
  version), default to [].
• Update get_schema() with the new field definition.
• Update _has_handle_reference(), _remove_handle_references(),
  _replace_handle_reference(), and get_referenced_handles() to cover every
  handle in super_event_list — so that deleting a super-event automatically
  removes it from all sub-events' lists.

gramps/gen/types.py

• Confirm EventHandle is exported (expected, no change likely needed).

Phase 2 — Database Migration

gramps/gen/db/generic.py

• Bump VERSION from (22, 0, 0) to (23, 0, 0).
• Add get_sub_events(handle: EventHandle) -> list[Event] — queries all events
  and returns those whose super_event_list contains the given handle.

gramps/gen/db/upgrade.py

• Add gramps_upgrade_23(): iterate all raw event records and append []
  (empty super-event list) to each serialized tuple. Standard no-op migration
  to bring existing data to the new schema shape.

Phase 3 — Editor UI

gramps/gui/editors/editevent.py

• Add a "Part of" section to the editor, below the existing Place row.
• Use a small embedded list widget with Add / Remove and Up / Down move buttons
  (the same pattern as PlaceRefEmbedList), since an event can belong to
  multiple super-events and the researcher may want to set a manual order when
  dates alone are not enough to distinguish sequence.
• Each row in the list displays the super-event's type and description; a
  button opens the existing SelectEvent dialog to choose one.
• At save time, check for cycles: walk the ancestor chain and reject if the
  selected super-event is already a descendant of the current event.

gramps/gui/editors/editevent.glade

• Add the corresponding label and list widget to the Glade layout.

Phase 4 — Events Tree View

gramps/plugins/view/eventview.py

• Add a toolbar toggle: Flat list (current behavior, unchanged) vs.
  Hierarchy view.
• In hierarchy mode, switch the backing model to a TreeStore instead of
  ListStore.
• Root nodes = events with an empty super_event_list.
• Children of a node = events whose super_event_list includes that node's
  handle.
• An event with multiple super-events appears under each of them (DAG display).
• Recurse to arbitrary depth.

gramps/gui/views/treemodels/eventmodel.py

• Extend the model to support the two-mode (flat / tree) display.

Phase 5 — Gramplets and Reports (deferred)

Narrative and graphical reports that iterate events are unaffected in Phase 1 —
they already show all events. Future follow-up work can:

• Indent sub-events under their super-event in individual event lists.
• Add a "Sub-events" section to the Event Details gramplet.
• Allow the Detailed Ancestor / Descendant reports to optionally expand
  sub-events inline.

---

File Change Summary

File	Change
gramps/gen/lib/event.py	New super_event_list field; serialize/unserialize; handle-ref methods
gramps/gen/types.py	Verify EventHandle exported (likely no change)
gramps/gen/db/generic.py	Version bump to (23,0,0); get_sub_events() method
gramps/gen/db/upgrade.py	gramps_upgrade_23() migration
gramps/gui/editors/editevent.py	"Part of" list widget with Add/Remove and cycle guard
gramps/gui/editors/editevent.glade	Widget row for "Part of"
gramps/plugins/view/eventview.py	Flat/Hierarchy toggle
gramps/gui/views/treemodels/eventmodel.py	TreeStore support for hierarchy mode

---

Pros and Cons of the Overall Approach

Pros

• Mirrors an existing pattern. The Place hierarchy already works this way.
  Gramps developers and users already understand placeref_list; super_event_list
  follows identical conventions, keeping the codebase consistent.
• Non-breaking. The new field is appended to the serialized tuple and
  defaults to []. All existing data is valid. No existing behaviour changes.
• Multi-hierarchy support from day one. A voyage event can belong to both
  an Emigration hierarchy and a Ship Journeys hierarchy without duplication.
• Minimal new abstractions. No new secondary-object class. No new database
  tables. No changes to the plugin system.
• Flat view preserved. The existing Events view is unchanged in flat mode.
  Users who never use sub-events see no difference.

Cons

• O(n) sub-event queries. Finding the sub-events of a given event requires
  scanning all events. For very large databases this could be slow. A secondary
  index would fix this but is deferred.
• DAG complexity in the view. Displaying a DAG (an event appearing under
  multiple nodes) in a GTK TreeStore requires duplicating rows, which can
  be confusing if not clearly signalled in the UI.
• Cycle detection cost. Checking for ancestor cycles at save time requires
  walking the chain, which is also O(depth). Depth is expected to be small in
  practice (single digits), so this is not a concern in practice.
• Report updates deferred. Existing reports will not show sub-event
  structure until follow-up work is done. This is a known gap, not an
  oversight.

[StoltHD]

Just a quick comment on the O(n) point in the cons section:

Gramps already maintains a relation table that effectively forms a graph
(person–event, event–place, event-citations, etc.).
For sub-event traversal, using this table directly avoids full scans and
reduces the complexity to the size of the connected component rather than
the total number of events.

There has been some discussion in other parts of the project about using
graph-based lookups for faster queries. Sub-events, as well as all existing
and future relation types in Gramps, fit naturally into that same pattern.

To really utilize the relation table for speed, graph logic and graph algorithms should be considered.

---

Semi-related comment:

One additional point worth considering:

The current relation table is very flexible, but it also has to carry all
relation types in a single structure. For performance and clarity, it may
be worth evaluating whether splitting the relation table into
object-specific relation tables (place-place, event–place, event–citation,
person–event, etc.) would simplify indexing and reduce query cost.

This would also make graph-based traversal more efficient, since each
relation type could have its own optimized index rather than sharing a
single large table.

---

Note: it might be I come with some more comments later, but as I read this Design Plan today, it is more or less picked out of my brain with a spoon... the Sun is shining right now...

[dsblank]

Just a quick comment on the O(n) point in the cons section:

Gramps already maintains a relation table that effectively forms a graph
(person–event, event–place, event-citations, etc.).
For sub-event traversal, using this table directly avoids full scans and
reduces the complexity to the size of the connected component rather than
the total number of events.

There is already an index on event handles, so not sure what the AI is talking about there.

There has been some discussion in other parts of the project about using
graph-based lookups for faster queries. Sub-events, as well as all existing
and future relation types in Gramps, fit naturally into that same pattern.

Yes, I having been following that, and in fact created the first "business logic" methods in the database layer that can be implemented by the database (such as SQL). But I stuck to current Grampic philosophies for this plan. I think we can talk about the basic implementation without requiring a re-write of the Gramps DB.

The current relation table is very flexible, but it also has to carry all
relation types in a single structure. For performance and clarity, it may
be worth evaluating whether splitting the relation table into
object-specific relation tables (place-place, event–place, event–citation,
person–event, etc.) would simplify indexing and reduce query cost.

Yes, I considered that. But I believe that if we do it, we should do it for all such relationships, not just for one in this plan.

[StoltHD]

I totally agree that the first step should stay true to existing Gramps logic.
Any graph logic, splitting of the main relation table, moving more of the frontend logic into the database, etc. can be done step by step – one concept, one new feature at a time.

The same principle as with the move from blobs to JSON: get the core model right first, then iterate on performance and structure when it becomes relevant.