[FEAT] - Convert consulting site blog machinery to match that of labs site

[bskinn]

What site is this for?

Quansight LLC

Expected behavior

No response

Actual behavior

Upon further internal discussion, we've decided that we want the consulting blog to be converted to use the same machinery as the labs blog. All posts are to be composed within the repo as Markdown files, and none are to be drawn from Storyblok.

Once the functionality of this PR has been implemented, all sixteen (16) of the blog posts currently in Storyblok will need to be migrated before the whole functionality+content package is merged into develop.

This change should be completed before work on #182 starts, so that the work of #182 can be smoothly applied to both the consulting and labs blog code.

We will need to decide whether we want an RSS feed to be generated for the LLC blog, analogous to the Labs blog. This will require fixing of the RSS machinery first, however, per #403.

How to Reproduce the problem?

No response

Anything else?

No response

[gabalafou]

I'm not sure why this needs to depend on #182. If #182 is done before this, then when we migrate the blog posts out of Storyblok and Wix, we'll just use the multi-author mechanism in place. If #182 is done after this, then we'll have to go back and fix whatever markdown files are already in place (as we will have to already do with several Labs posts).

[bskinn]

Hm. I guess it depends on where the code for this lives. I was picturing that as part of implementation of this issue, the blog post handling code would get moved to the shared code, and then #182 would be a single fix in that shared code.

[bskinn]

And that the addition of multiple authors to the individual blog posts would be a series of PRs following the merge of #182

[bskinn]

I'm not strongly tied to this, though -- and I haven't thought it through in careful detail.

[bskinn]

Notes from the 2022-09-19 sprint planning call between @bskinn, @kherma and @tokordys:

• Quansight wishes to keep the "Post Types" functionality for the Content Library (/library) page, as we will be adding content of additional types (e.g., 'case-studies' and possibly 'notebooks') to /library in the near- to medium-term.
• Definition of the "Post Categories" for each post should be relocated out of Storyblok and into the Markdown header/metadata, in the same fashion as for the Labs blog
• For all /library "Post Types" other than blog, if it is not prohibitively difficult to implement, the content for these Library Link items should continue to be defined in Storyblok.
  • If it proves too difficult to implement in this way, it will be fine for the Library Link items to also be migrated to where they are defined in the repo code.
• IF the Library Link (non-blog) content items continue to be defined in Storyblok, there is no need to bring the concept of "Post Type" into the handling of the blog posts in the repository code. This is because the only post type that will be migrating to the repo code are the 'blog' type
  • If it proves necessary to also move the definition of the Library Link items into the repo code, however, then some mechanism for assigning Post Type from within the repo will have to be devised.
• Quansight will be responsible for migrating the existing blog posts to the new in-repo Markdown format ~concurrently with the Naturaily functional work, so that the migrated blog content is ready for merge to develop at the same time as the code.

[bskinn]

@kherma -- I've been poking around the Library section of the code and of Storyblok, and the Post Categories data is used by all of the items in the Library: both blog posts and Library Links.

Would it be possible to retrieve Post Categories from both the Storyblok data source and the metadata of the blog posts housed in the repo? It seems like that might be less work than migrating both the Library Links and the blog posts to live in the repo...but I'm not sure.

From a content-creator-interaction perspective, I think it makes the most sense to keep the Library Links defined in Storyblok, if it's not too hard to achieve. Library Links are most likely going to be added only by Marketing, and Storyblok is the more natural interface there.

[bskinn]

Reproducing here the conversation between @kherma and me in order to attach the discussion to the issue.

[bskinn]

kherma:

Okay, so the question is, whether or not you would like to assign multiple authors to library links e.g. webinars, videos, notebooks etc or is all just about Blog Posts themselves? Because currently the author field is of type single-option so in order to add multiple authors it requires multi-option type.

bskinn:

At this point, let's do whichever option doesn't require a change to the schema. I assume that's retaining single authors on library links.

More generally, I think single authors on the library links should be fine, long term. I can't imagine many situations at the moment, where those would require multiple authors.

kherma:

Got it.

[bskinn]

kherma:

And also to confirm. The Idea is to leave library related datasets in Storyblok as they are used by library links. So after converting blog posts to markdown, categories in meta section should match those from datasets? Or you mean that the categories in posts meta section could be anything and after posts are fetched all the unique (not present in datasets) categories will be added to dropdown.

bskinn:

Yes, as long as it's not prohibitively difficult, I'm picturing the following configuration:

• Category/categories for each blog post in Github/Markdown are defined in that post's metadata
  • These categories don't need to be defined in the Storyblok Data Source
• Category/categories for each Library Link item are selected from those defined in the Storyblok Data Source
• At site build, the set union of the following is what gets populated to the library "Category" dropdown:
  • The categories defined across all of the Github/Markdown blog posts' metadata
  • The categories defined in the Storyblok Data Source
• Addendum: The categories populated into the "Category" dropdown should be sorted alphabetically, if they're not already

kherma:

👌

[bskinn]

kherma:

So for now there is just one last thing to consider. The one main remaining difference between LLC library and Labs blog is how categories actually work. On Labs blog every category is just a string but on LLC Categories in Datasources are stored as Key-Value pair. So the question is how do you prefer to assign one of the existing datesource’s category to markdown blog post. Just as reminder: Key is what should be displayed in Category Dropdown and the Value is used to filter Library Tiles and also as Value of query param in url. I thought about quickest solutions:

(1) Use Key or Value or either of them from datasource as category in blog post meta section and compare it against categories fetched from Storyblok. In my opinion using Key (display name) makes the most sens simply to keep things consistent as any category added to meta of blog post should be defined as you desire to display it in dropdown and also that way will match te workflow of Labs Blog.

(2) Another option might be to simply rename all the values in datasources to match the Key... which I’m not a huge fan of because it’s actually same as skipping the value for the key everywhere.

Also found a small bug. The props passed to library dropdown as display names are actually the values from datasource - they should be Keys. I couldn’t find issue for that one (might need to double check) but as it’s 5 min. fix I can simply do it with current task.

bskinn:

Ahhh, I had wondered about why there were two values visible. In my non-admin Storyblok view, the keys are locked but I can change the values. I was fuzzy on what the difference was between them, and key-value pair makes perfect sense.

---

In my opinion using Key (display name) makes the most sens simply to keep things consistent as any category added to meta of blog post should be defined as you desire to display it in dropdown and also that way will match te workflow of Labs Blog.

I agree, pairing the Storyblok Data Source keys with the Markdown metadata categories makes the most sense, for all of these reasons you state.

However, if it's feasible, I think we probably need to keep the entire key-value paradigm for the Consulting blog, even though that will deviate from how the Labs blog works.

This is because we really do need both the keys and the values:

• Since the value is used for the query parameter in the URL, we really want to avoid spaces, so a slugified version is preferable.
• But, for visual display (like in the Categories drop-down), we would want categories with spaces to render those spaces, and not as slugified (like in the bug you noticed)
• The ability to manually choose the value/slug for each category is valuable, because (presumably) we could choose shorter slugs for things. (E.g., for a key of "Dashboards/Visualization", we could choose a value of "visualization" or "dashboards-viz" or whatever.)

I noticed that the Labs site uses a categories.json file to define allowed categories. We could maybe have something similar for Consulting, just instead of being a list of categories, it's a mapping of key-value pairs, analogous to those in the Storyblok schema for the 'Post Categories' Data Source.

Consulting blog posts would then be required to choose only from the keys of this mapping in categories.json for the categories in their metadata.

The set union of the categories for blog posts (from Github) and Library Link items (from Storyblok) would then be the union of the two mappings. One wrinkle here is that we would have to decide how to handle key collisions. My inclination here would be for the build to error in the case of identical keys with differing values. This way, it would surface that collision in an obvious way, and prompt us to make a conscious decision about how to reconcile the values.

I also like the prospect of having a separate 'source of truth' for the categories for the blog, which the categories.json file would provide. (This would provide an automatic double-check against typos in the categories entered into blog post headers.)

---

Yes, please do fix the bug in populating the dropdown, to use the keys instead of the values (and, also to alphabetize the entries within the dropdown). It looks strange having the hyphenated, slug-like items there.

I'll go ahead and create a formal GH ticket for this issue and attach it to the sprint. We should document these details of the work.

[bskinn]

@trallard @noatamir (cc @gabalafou):

Downstream of the above discussion, @kherma pointed out that it would simplify the codebase if both the Consulting and Labs blogs were implemented using a key-value pair for categories: identical machinery could then be shared by both sites.

As best I can figure, there would be ~no changes to the Labs blog authorship workflow: the post categories would still be selected by entering category names (the keys, of the key-value pairs) in the metadata header.

The only change would be in definition of new categories: both the key (category name) and the value (category slug) would have to be defined. (As a one-time task, the slugs for existing categories in categories.json would also have to be defined as part of the implementing PR.) This seems like a trivial increase in effort, and has the benefit of allowing customization of the slug (used in the URL to filter the posts shown on the main blog page).

It seems like the best course of action to me. Thoughts?

[trallard]

As best I can figure, there would be ~no changes to the Labs blog authorship workflow: the post categories would still be selected by entering category names (the keys, of the key-value pairs) in the metadata header.

There will be no change in the workflow, but that would mean opening PRs to fix the existing posts + opened PRs. So I would rather not have this changed now that we have managed a steady workflow and cadence for the migration, and considering that we have a big backlog of PRs to review and merge.

The only change would be in the definition of new categories: both the key (category name) and the value (category slug) would have to be defined. (As a one-time task, the slugs for existing categories in categories.json would also have to be defined as part of the implementing PR.)

This makes it sound that consulting and Labs would share categories, in which case this is a hard no for me.
Each site has different tones and audiences, and our blog's categories reflect what we focus on as strategic areas within Labs. The Consulting ones do the same, but for the consulting site. We - in Labs - do not need a mixed media library, so having key:value pairs is overkill for us.

While the underlying workflow + system should be the same for both sites - the actual implementation should be tailored to the use cases and whatever makes sense to each site and its audiences. So my vote goes to not changing the Labs workflow or system from its current state at all.

[bskinn]

So my vote goes to not changing the Labs workflow or system from its current state at all.

@kherma has already implemented the Consulting blog machinery conversion in #577 without touching anything in the Labs portion of the codebase, so this is what we will be doing.

---

For completeness, and for possible future reference:

... but that would mean opening PRs to fix the existing posts + opened PRs.

This is not correct. There would be no changes required to Labs blog posts at all.

The only Labs staff action would be to review the proposed new values for each category in the key/value mapping introduced in https://github.com/Quansight/Quansight-website/blob/develop/apps/labs/posts/categories.json. These new values would be the slug-like representations of the category names, used e.g. in filtering blog posts for display.

For example, the current "Developer workflows" category might become {"name": "Developer workflows", "value": "developer-workflows"} or {"name": "Developer workflows", "value": "dev-workflows". (I could see renaming the value key to slug, or similar.) Posts currently tagged with Developer workflows would continue without modification to be tagged with Developer workflows. The Labs staff decision for this tag would be between the developer-workflows, dev-workflows, or whatever other slug representation. Similar decisions would be required for the remainder of the tags.

This makes it sound that consulting and Labs would share categories ...

This is not correct. They would be independent.

Labs categories are defined in https://github.com/Quansight/Quansight-website/blob/develop/apps/labs/posts/categories.json

Consulting categories are defined in https://github.com/Quansight/Quansight-website/blob/feat/QUAN-14-llc-blog-machinery-conversion/apps/consulting/posts/categories.json (not yet present on develop)

This independent definition would hold even if the Labs categories were converted.

[noatamir]

@bskinn for my understanding: where would we use the new categories slugs in the Labs side?
They are defined in the json and then used where?

[bskinn]

@noatamir:

bskinn for my understanding: where would we use the new categories slugs in the Labs side?
They are defined in the json and then used where?

@kherma may know of more points of use, but the primary one I'm aware of is in the query portion of the URL for a filtered view of the blogs listing.

Viz.:

1. Navigate to labs.quansight.org/blog

2. Scroll down and click a category to filter against; e.g., "Developer workflows"

   

3. Examine the address bar. Currently a direct slugification of the category name is used in the query in the URL, without even a lowercase coercion:

   

The new setup would permit (at minimum) custom selection of the slugs used as the URL category= query parameter.

It would also reduce at least some of the work required for future changes/augmentations to blog-related site machinery, because more of the Labs and Consulting code (again, to emphasize, not the categories themselves) could be directly shared.

[noatamir]

So, the slugs would replace the current categories in the links you mentioned?
Does it mean a breaking change for links to current categories?

To clarify, I have no idea if anyone externally is linking to our categories currently, so I am not necessarily concerned. But there are a few older blogs where we were considering linking to the categories as part of the migration (they had links to defunct categories in the old website).

I am in general in favor of making this change as I understand that it doesn't cause any significant workflow overhaul to Labs, and it will reduce complexity on future maintenance of both websites. Let's see if Tania mentions some other difficulty that we haven't foreseen?!

[bskinn]

Does it mean a breaking change for links to current categories?

Excellent point, I hadn't thought of that at all. Yes, a change to the slug (not sure about letter-case-only changes) would break incoming and intra-site links to categories: such links would be pointing to a now-nonexistent category, and so the page would list no posts.

[bskinn]

@trallard / @noatamir -- do we need to adjust imgbot's configuration to also optimize the images for the Consulting blog posts? (/apps/consulting/public/posts/...)

[trallard]

Nope it should pick them up automatically