One catalogue, several front ends

[Maximilian-Nesslauer]

@SafeShows shipped ksamods.gg , with a public index repository at ksamods-gg/index carrying real listings and stamped releases in something close to the RFC 0031 shape.

It also means there are now two indexes, and RFC 0033 did not anticipate that.

The RFC says that a central index all projects work from is the goal.

Please reference the numbers so the thread stays followable.

1. The index can not exist twice

The content id namespace needs exactly one arbiter.

For mods the id is the folder name, because Mod.MakeUsing overwrites whatever mod.toml declares (see research/ksa-mod-loading.md), and RFC 0031 extends that namespace across every content type.
Two catalogues that both hand out ids will eventually hand out the same one twice and cause conflicts.

2. Who is the Authority Decision

RFC 0033: git is the record.
Two static repositories hold the authored documents and the stamped releases, humans and a scheduled watcher write through checks, clients read one snapshot artifact.
No server anywhere, which is what RFC 0025 requires, and mirroring the whole catalogue is git clone.

ksamods.gg today: the server is the record.
The site holds the data and an exporter rewrites ksamods-gg/index every half hour or so; every commit there is authored by "ksamods exporter".
Nothing lands by pull request, so the repository is a report of the database rather than the thing the database was built from.

I would take the git-of-record one, for the reason RFC 0025 gives. We dont want to rely on someone sponsoring infrastructure somewhere we could loose access to at any time once they cant sponsor it any more.

3. The format delta today

@SafeShows said the index follows the RFC, and structurally it does but two gaps are not cosmetic:

• No game_min anywhere, neither on the listings nor stamped into any release. It is required by RFC 0031, and without it RFC 0017 evaluates every entry as Unknown.
• No [releases] section on either listing, so nothing in the index says where the next release comes from or which host is the authority.

Plus a medium one: index.json is a counts manifest rather than the snapshot artifact RFC 0033 defines as the client contract, and moderation-log.json is an append-only action log where RFC 0033 has a steward-owned index-status.toml with delisted, disputed and retracted.

The rest is small and mechanical: listings are JSON at a lowercased path instead of TOML at listings/<id>.toml, StarMap is typed mod instead of mod-loader, release files carry no listing snapshot block, download.content_type is whatever the GitHub asset API reports, name is the repository name instead of the display name, and builds.json is empty where KSA-CKAN-meta has a working master-server poll to copy.

4. What I would propose

The canonical index stays what RFC 0033 describes, in the KSAModding org: two static repositories, a watcher, publish-time checks, one snapshot artifact, mirrorable by anyone.

ksamods.gg becomes the WebUI front door, which is the half RFC 0033 is weakest at:

1. The renderer. The snapshot is one fetchable document with everything in it, so a site that reads it needs no database to browse, search and link content, and stays correct with no sync logic.
2. The listing and publish flow. RFC 0033 assumes an author runs a CLI or opens a pull request by hand, and realistically most will do neither. A web form that authenticates with the author's own GitHub account, validates, and opens the pull request for them is strictly better, and it is what @MrJeranimo proposed in the RFC 0031 thread and I agreed to there.

What makes this safe is already in RFC 0033: the index never trusts a submitted fact.
The pull request path re-downloads the archive, recomputes the hash, sizes and install root, re-runs the dependency merge, and rejects anything that disagrees.
So a hosted service can produce submissions without anyone having to trust the service, and the day the site is down the catalogue is unaffected.

Everything that genuinely needs a server, accounts, comments, votes, social signals, stays on the site and out of the index, which is where RFC 0025 already puts it.

5. What each side would have to do

Us, and this is the larger half. finally create the index repositories, the watcher, the checks.

ksamods.gg. Point the publish flow at the canonical repositories of this org instead of at an export of its own database, and close the gaps in 3.

[Maximilian-Nesslauer]

My position, plus a correction to something I said.

What the RFCs are missing is a defined role for the services that live next to the index, so I would add one RFC on top of RFC 0033 rather than rework it.

6. Three options, and the one I would take

A. Git stays the record, and every service is a derived view with a stated contract.
A derived view, meaning the ksamods.gg database, a search node, or a CDN cache, builds only from the canonical index, is reconstructible from zero, declares its provenance, and keeps service-local data such as accounts, votes, and comments out of the index, which is where RFC 0025 wants them anyway.

A front end additionally produces submissions: it authenticates the author with their own GitHub account, validates locally so the author gets a fast answer, and then opens the normal pull request against the canonical repositories on the author's behalf.
This is what I propose.

B. A cloud database is primary and git is an export.
This is the ksamods-gg/index shape today, where every commit comes from the exporter.
Good latency and easy search, but the write path dies with the service, and the export is a report of the database rather than the thing the database was built from.
It is completely fine as a front end's internal architecture. It is not what I would make canonical, because it depends on somebody hosting the database and it is less transparent than A.

C. True multi-master, several indexes syncing each other in both directions.
This is the one I like least, although it sounds the most attractive.
First come, first served on a global case-insensitive id namespace is not decidable with two equal writers: two indexes can grant the same id to two different people inside one sync window, and there is no correct merge afterwards.
The id namespace needs exactly one arbiter, which is the whole folder-name argument behind RFC 0031 and RFC 0033 (Mod.MakeUsing overwrites whatever mod.toml declares).

The part of A I like most is that git already is the sync protocol.
A mirror needs no event bus and no SQL export. It runs git pull, or subscribes to push webhooks with polling as the fallback.
For ksamods.gg that means turning the exporter around, git to database instead of database to git, and the database becomes a cache that can be rebuilt from git at any time.

The trust question is already answered by RFC 0033: the index never trusts a submitted fact.
The checks re-download the archive and re-derive the hash, the sizes, the install root, and the dependency merge, and reject the submission when any stamped field disagrees.
So a hosted service can produce submissions without anybody having to trust that service, and on the day the site is down the catalogue is unaffected.