Add frab-compatible JSON schedule

[Kunsi]

• Move Frab XML exporter to class frab_exporters.py
• Move some common code to parent class FrabExporter
• Add Frab JSON exporter
• Fix or disable tests
• Add Frab JSON exporter to site export
• Address issues mentioned in schedule/feeds: make json feed schema compatible #1612

Did i miss anything?

[Kunsi]

We could improve this by explicitely setting colours here for each proposal type, but i feel this is good enough(tm) for now.

[wlcx]

I wonder if it would be better to just have a constant list of a nice palette of suitably distinct colors and pick from that in alphabetical order of the slug, hashing as a backup if we have more than in the list? Given the number of CFP types is (fairly) fixed and small, and distinctiveness of the colors from each other is the main point of having them?

[Kunsi]

Yeah, after testing this (see mattermost ~video-infobeamer) i've noticed that "youth workshop" and "talk" are awfully similar. Should i just grab 2024s colours off info-beamer?

[wlcx]

Perhaps - I can't remember if they were manually set or not! I don't think the exact colors really matter at all, we don't (afaik) have color schemes for types of content. so i'd be inclined to just grab a suitably contrasting palette from anywhere and use that

[Kunsi]

They were! I imported them for now.

[marksteward]

A lot of this code was rewritten as part of the content refactor. In particular, we now have recurring schedule items so a workshop might have multiple start times and venues. I'm going to convert this to draft and make it match after the content refactor lands, which should be in the next few days.

[Kunsi]

@marksteward oh, i thought the CfP refactor was already done, otherwise i would not have created the PR (and instead waited for the refactor to finish). Sorry about that.

[Kunsi]

filtering based on village_id doesn't seem to work, other than that i don't see any remaining bugs or features i'd like to have.

[marksteward]

@Kunsi not a problem, I'm happy to do the required changes for this too, but probably not before the refactor has landed.

[Kunsi]

Random thought that floats around in my head: if events happen multiple times AND they shall be recorded, we need different slugs and different GUIDs for them.

[marksteward]

Yep, the way I'm doing it to work with Frab is I flatten the hierarchy so instead of a single ScheduleItem with multiple Occurrences, we get multiple ScheduleItems each with one. The slug and GUID are then generated using the occurrence_num field that exists on Occurrence.

(I've also changed the GUID logic to not look at our slugs, because those depend on the title of the schedule item so can change.)

[Kunsi]

I didn't find anything in the source code, please let me know if i missed anything.

[wlcx]

I'm not at all spooled up on the recent website changes so not the best placed to do an in-depth review, but I think this is a reasonable thing for us to have :)

[wlcx]

I wonder if it would be better to just have a constant list of a nice palette of suitably distinct colors and pick from that in alphabetical order of the slug, hashing as a backup if we have more than in the list? Given the number of CFP types is (fairly) fixed and small, and distinctiveness of the colors from each other is the main point of having them?

[wlcx]

Lol, github just straight up.. didn't show me your discussion before I hit submit on the review?

[marksteward]

On the topic of slugs, what are they actually used for downstream of us? Does something generate URLs with them, and are they expected to unique for each occurrence of an event?

[Kunsi]

On the topic of slugs, what are they actually used for downstream of us? Does something generate URLs with them, ...

Yes, atleast media.ccc.de uses them for generating URLs for the released videos. I'm not aware of anything else off the top of my head

and are they expected to unique for each occurrence of an event?

Yes, the spec even enforces that: https://github.com/voc/schedule/blob/master/validator/xsd/schedule.xml.xsd#L86-L89 (also ID and GUID need to be unique)

Theoretically we could drop the ID from the slug (it's not required, just often included to ensure it is unique).

[marksteward]

I was actually wondering if we need to drop the title from the slug. If the title of a schedule item changes, is it OK for the slug to change too? Will that mess up any existing pages or files on c3voc's side?

I'm going to distinguish our ScheduleItem.slug (used in our URL generation) from Occurrence.c3voc_slug, which will be unique for a specific occurrence of an event. So I think that'll give us something like:

ScheduleItem

id = 123
title = "The foo bar"
slug = "the-foo-bar"

Permalinks:
  https://www.emfcamp.org/schedule/2026/123
  https://www.emfcamp.org/schedule/2026/123-the-foo-bar (canonical)

Occurrence

occurrence_num = 1
c3voc_slug = "emf2026-123-1-the-foo-bar"

Permalinks:
  https://www.emfcamp.org/schedule/2026/123/1
  https://www.emfcamp.org/schedule/2026/123-the-foo-bar/1 (canonical, but may redirect to ScheduleItem page)

(Our slug isn't required to start with a letter)

Event

id = 123
guid = uuid5(NAMESPACE_URL, "emf2026-123-1")
slug = "emf2026-123-1-the-foo-bar"

Permalinks:
  https://media.ccc.de/v/emf2026-123-1-the-foo-bar

There's a note about id on the schema:

    <!-- legacy numeric identifier from pretalx, might be non unique when submission has multiple time slots, c.f. https://docs.pretalx.org/developer/concepts/#schedules-and-slots -->

Our Occurrences are pretalx's Slots.

[Kunsi]

If the title of a schedule item changes, is it OK for the slug to change too?

Yes, that's the plan. The slug should always match the title to avoid confusion.

There's a note about id on the schema:

Yeah, and it always messes up everything when that happens, so it's advised to have them unique. What i'd probably do is do something like frab_id = event['id']*10 + occurence_id (assuming events don't occur more than 9 times).

What you proposed re c3voc_slug sounds like a plan, though for easier reading i'd omit the occurence_id from events that only happen once.

[marksteward]

I can't easily omit occurrence_num from events that only happen once because our slugs can start with a number. Titles like 1-hour-computing-degree or 5-things-we-can-all-learn-from-flaming-pianos would be ambiguous, and putting a different delimiter in will probably make readability worse.

I'll use 100 * schedule_item_id + occurrence_num for now, I don't love it but it's probably the sensible thing to do.

[Kunsi]

because our slugs can start with a number.

yeah, i can see this would be confusing

I'll use 100 * schedule_item_id + occurrence_num for now,

In case this wasn't clear, we only have to do this for the id - in slugs we can do emf2026-123-1-foo-bar-baz just fine.

[marksteward]

I've rebased this onto the CfP refactor at #1909, and added a FrabExporterFilter to keep that bit more type-friendly.

Given this code already uses the DB classes, I wonder if we should pull data directly from the models instead of relying on the schedule/data.py stuff?

[marksteward]

The Frab schema says slug must match [a-z0-9_]{4,}-[a-z0-9\-_]{4,}, so only one hyphen allowed 🙃. For now slug will use the hacky ID, but multiplied by 1000 to make it 4 digits, so emf2026-123001 for the example above.

[lukegb]

The Frab schema says slug must match [a-z0-9_]{4,}-[a-z0-9\-_]{4,}, so only one hyphen allowed 🙃. For now slug will use the hacky ID, but multiplied by 1000 to make it 4 digits, so emf2026-123001 for the example above.

Hyphen is in the second character class?

[marksteward]

Thank you, fixed!

[marksteward]

Rebased and merged as 9395d38