Enhancement: "data collections"

[phillipadsmith]

Howdy Jekyll Folks,

(Apologies if this idea isn't fully baked, or the right place to ask -- just want to get it out of my head while I'm thinking of it...)

I've been working a bit with collections and data files recently and I'm curious if anyone else would find the idea of a "data collection" to be useful.

A data collection would simply be a collection where the source for the output of the collection is a data file. For example, where a CSV (or JSON, or YML) file like this could result in the same output that is currently generated by these files.

Thinking it through a bit, there would likely need to be a way to specify what layout to use to output the collection. Perhaps that could be done in the configuration file, e.g.:

collections:
  toronto_city_council:
    output: true
    data: toronto_council.csv
    layout: candidate.html
    permalink: /:some/:val/:from/:data/:file.html

The use case, as demonstrated here, is that I'm often generating hundreds of stub files for a collection from the same CSV that is later used to populate much of the data for that collection's output.

Let me know if any of this makes sense. And, if so, what would be helpful in thinking it through further.

Phillip.

[budparr]

If I understand you, I think it's a great idea. I have a use-case of a website for a publisher where I have a collection of books, each with a lot of metadata that I'd rather keep in data files, but need the attributes of collections, like, for instance, each being output to its own page. Of course, I can use both and join them in the templates, but it might be easier and cleaner to just generate the entire thing in a data file.

[phillipadsmith]

@budparr You are understanding me correctly. You and I both have essentially the same use case.

[pathawks]

Why not use Collections as they currently are, and just put all your data in the front-matter?

[phillipadsmith]

@pathawks Have a look at the CSV that's referenced above. With that many rows and columns, it's just easier to work with a CSV file for updates. Updating the .md files manually would be a big PITA and updating them with a script just feels wrong.

There's great data support in Jekyll now (CSVs, finally!) and it makes more sense (to me, anyway) to use that support instead of the kludge that I'm currently using to make stubs for each row in the CSV in a collection.

Maybe I'm thinking about it all wrong...?

[pathawks]

Oh

So you want a one-to-many relationship between input (your CSV file) and output (the generated pages)?

[phillipadsmith]

So you want a one-to-many relationship between input (your CSV file) and output (the generated pages)?

Yes, exactly. :)

[phillipadsmith]

@parkr Thoughts on the proposal above?

[parkr]

At this point I'm not for it. Let me think on it some more.

[parkr]

It sounds like you want collections but with multiple layouts, there's another issue open for that already, #3041.

[phillipadsmith]

Hi @parkr It's not really the same, no.

Basically, the proposal is for collections that are built from a data file, not markdown files.

I think that's quite different, no?

[parkr]

Yaml front matter should solve this need, it would seem. If there's no content, it's just a data file anyway :)

[phillipadsmith]

The challenge, however, is when you want to have an page output for hundreds of items. In the case of this project, I'm generating the .md files for the collections from a CSV.

That step would be unnecessary if it was possible to just indicate that a collection was associated with a data file.

This is a common pattern / use case in other static site generators.

The addition of the native CSV processing in Jekyll was a huge step forward, IMHO. The addition of collections connected to a data source like a CSV (or the CSV output of a Google Spreadsheet) would take non-technical copy/content editing of a Jekyll site to another level, i.e., certain collections would be as easy to update as editing a spreadsheet and re-generating the site.

You can see another use case in this project of mine where each collection of people (often hundreds of people) is actually just data from a CSV file.

This is a common pattern for "news apps," or what are essentially online databases that don't update enough to justify a dynamic approach.

[davidrleonard]

Not to resurrect a dead thread there, but wondering if any more thought was given to this? This is also a big need for me in designing large Jekyll sites that have a lot of pages with very similar content (i.e. bio pages for hundreds of people). I'd love to be able to manage this data elsewhere, like a Google Sheet, and export it to a single file then build my pages out from there.

[envygeeks]

This ticket is being locked. If you have an issue (or feature request) please file a new ticket instead of practicing the art of necromancy. It has hard to maintain actual issues against comments if this happens.