Add ocaml.org Hand-picked RSS feed

[cuihtlauac]

DO NOT MERGE

This is a preliminary and hacky attempt at addressing issue #1203.

We have a manually created, edited and committed asset/ocamlorg-handpicked-feed.xml file containing one entry per foreign feed entry. This is served as-is.

ocaml.org recursively becomes its own feed source, adding https://ocaml.org/ocamlorg-handpicked-feed.xml to data/planet-sources.yml

Updating is where it becomes ugly.

1. Update asset/ocamlorg-handpicked-feed.xml
2. Push online
3. Scrape
4. Push online a second time

Going further:

• Patch river to make it able to fetch a source from a local file (this would remove duplicate push)
• Generate asset/ocamlorg-handpicked-feed.xml from a Yaml data file

Tests results:

• make scrape
• make start
• Browsing

[tmattio]

Happy to review, but I'm not sure what we're trying to achieve here, I suspect this isn't the right issue in the description. Can you give more context on the problem?

[tmattio]

Ok, reading the code I think I understand. Instead of adding an RSS feed, could we simply add the posts in data/planet/ocamlorg/ without content, only a redirect to the blog posts?

[tmattio]

Or even better, add new directories stackoverflow and backendbanter in planet/ with the posts without adding an RSS feed source.

[cuihtlauac]

@samoht suggested to have a podcast/video section.

Leaving aside deciding whether they go in /blog or elsewhere, a related issue is: we probably don't want to aggregate whole feeds which only contain a few ocaml-related posts (example: stack overflow blog).

I propose to have a mean to pick and choose individual posts from various feeds. This selection would be

• exposed as a standalone ocamlorg feed
• merged into /blog (or some other section TBD)

[tmattio]

@samoht suggested to have a podcast/video section.

Leaving aside deciding whether they go in /blog or elsewhere, a related issue is: we probably don't want to aggregate whole feeds which only contain a few ocaml-related posts (example: stack overflow blog).

I propose to have a mean to pick and choose individual posts from various feeds. This selection would be

• exposed as a standalone ocamlorg feed
• merge into /blog (or some other section)

Picking individual posts (or podcasts) for the planet makes sense. I'm am not sure about maintaining a local RSS feed for handpicked posts that is only used for scraping. The posts will have to be added to that RSS feed manually anyway, so might as well add the markdown files in data/planet/ directly, wdyt?

[cuihtlauac]

I'm am not sure about maintaining a local RSS feed for handpicked posts that is only used for scraping.

I'm not either :-) Although I see the value of subscribing to a feed which manually aggregates individual posts from various sources. That's curated content.

The posts will have to be added to that RSS feed manually anyway, so might as well add the markdown files in data/planet/ directly, wdyt?

The trouble is we may not have access to the markdown. That's why I'd rather reduce manual editing to merely copying a URL into a file.

[tmattio]

The trouble is we may not have access to the markdown. That's why I'd rather reduce manual editing to merely copying a URL into a file.

We don't need to have the content, for instance, the StackOverflow post in the RSS feed can be a new file in /data/:

# data/planet/stackoverflow/for-those-who-just-dont-git-it.md
---
title: For those who just don’t Git it
description: Pierre-Étienne Meunier, creator and lead developer of open-source version control system Pijul, joins the home team to talk about version control, functional programming, and why OCaml is a source of French national pride.
url: https://stackoverflow.blog/2023/05/23/for-those-who-just-dont-git-it-ep-573
date: 2023-05-23T21:04:40-00:00
preview_image: https://149351115.v2.pressablecdn.com/wp-content/uploads/2022/03/blog-podcast-relaunch-1.png
featured:
---

I think this is basically the same amount of manual editing, but doing it in markdown removes quite some complexity from the process. I'd also suggest we wait to see if doing this manually becomes a problem before we try to automate it. I suspect we won't be adding new blog posts manually very often.

[cuihtlauac]

You're right. This is easier your way. So basically, it's just a matter of editing CONTRIBUTING.md, that's great.

[tmattio]

One issue with this is that asset/feed.xml won't contain the manually added posts, since it's an aggregate of the XML files at planet/*.xml. We should change that to make feed.xml an aggregate of the markdown instead, and remove the .xml files in planet/, I don't think they are useful, so they are just adding noise.

[tmattio]

cc @davesnx I added a couple of your blog posts here - they are really great btw, I especially liked the React in OCaml one!

[sabine]

One issue with this is that asset/feed.xml won't contain the manually added posts, since it's an aggregate of the XML files at planet/*.xml.

I recently realized that a better way to do this is to generate feed.xml in ood from the markdown files. Then we don't need to cache any XML feeds either.

[tmattio]

I recently realized that a better way to do this is to generate feed.xml in ood from the markdown files. Then we don't need to cache any XML feeds either.

You mean in ood-gen or ocamlorg_data? if the latter, one possible downside is the cost of assembling the feed at runtime

[sabine]

You mean in ood-gen or ocamlorg_data?

ood-gen because, indeed, there's no point in doing it at runtime.

[tmattio]

Then we're in agreement, that's what I was suggesting above 🙂

We should change that to make feed.xml an aggregate of the markdown instead, and remove the .xml files in planet/, I don't think they are useful, so they are just adding noise.

[sabine]

that's what I was suggesting above 🙂

Skim reading comprehension fail on my side. Yes, we're on the same page.

[tmattio]

Let's merge as-is and make the changes to generate the RSS from the markdown in another PR

[sabine]

Sounds fine to me. Incremental progress is good.