Seeking Input: Pattern "Collection" Metadata/Overrides

[lyzadanger]

Hullo!

The way that things are shaping up, drizzle will continue to treat page frontmatter similarly to fabricator—e.g. you can override the default template by using a frontmatter property. This is straightforward because there is a one-to-one relationship of page source files to html output files.

With patterns, it's a different relationship. Any subdirectory of the patterns directory with any immediate-children pattern files is considered a collection and an HTML page will be generated for that collection (patterns subdirectories with no pattern files in them—e.g. a directory holding only more directories—will not cause a page to be generated).

Thus, for pattern collections, there is no current single point of metadata if one wanted to accomplish something like overriding the default template used to generate pattern-collection pages. There may be also other metadata we wish to associate with a collection of patterns in the future.

One plausible way to manage this would be to have an optional file in a given pattern directory that could contain this metadata. Naming options for this file include:

• _index.md
• collection.json or collection.yaml

...or really any number of things.

On the one hand, this feels a tad like overengineering as the only immediate application I can think of is overriding the default pattern-collection page template. But I feel like not thinking this through could leave us in the lurch later.

Any thoughts or ideas on this?

/cc @mrgerardorodriguez @erikjung @tylersticka @saralohr @nicolemors

[tylersticka]

It's hard to say because I can't think of a real-world (not hypothetical) scenario where I'd want to do this. Usually I base my recommendations on how to accomplish something based on my experience trying to do it, but in this case I've got nothing. 😕

[lyzadanger]

@tylersticka But thanks for your input! That's valuable, too. Maybe this really is a feature to worry about later.

[lyzadanger]

Another example, if it helps: pattern collections as of now will always be given a name that is derived from the directory name, e.g.

• Patterns in a directory called toes would be part of a collection called Toes
• Patterns in a directory called 01-fingers would be part of a collection called Fingers

Perhaps that's OK, but I thought I'd raise it.

[tylersticka]

Hmm, okay. In that case, I think I'd lean more toward the collection.{yaml,json} way of doing things maybe?

I dunno. Maybe it makes more sense to keep the formatting consistent with the patterns themselves. Or maybe that's more confusing because they differ.

Yeah, I'm talking myself out of every opinion I start to have. I'm gonna bow out of this one. 😅

[erikjung]

I'm fine with the collection.yml approach.

[lyzadanger]

FYIses, before I forget...I'm leaning toward using the collection.yml functionality to support custom ordering of patterns. The filesystem-prefixing convention is making me crazy and it makes munging nested objects rickety. Anyone got a strong rejection feeling about that? :)

[tylersticka]

I'm leaning toward using the collection.yml functionality to support custom ordering of patterns.

I'd rather there be an order property in individual pattern frontmatter.

Also, if we do move pattern order out of the filename and into frontmatter, I think we should do the same for hidden patterns... using a frontmatter property instead of a leading underscore.

Basically, I'm fine with ordering via filename or via frontmatter properties, as long as we're consistent.

[lyzadanger]

The reason I'd argue for using a collection.yml versus individual patterns is so that you don't have to open up each pattern to edit the front matter any time you want to adjust ordering. That is, you can see the ordering for the whole collection in one place. As the metadata involved applies to the collection versus the individual pattern (the pattern's order value doesn't mean anything when the pattern is on its own), I still think it makes sense.

Would like to get other opinions... @erikjung @mrgerardorodriguez @saralohr @nicolemors ?

Also, if we do move pattern order out of the filename and into frontmatter, I think we should do the same for hidden patterns... using a frontmatter property instead of a leading underscore.

I think this can be done! Though again I'd almost argue for the collection.yml being the right place for this—the patterns are hidden vis-a-vis their collection, but not overall (IIRC the requirement was to hide them from listings but still have them be available for partials and data?).

[tylersticka]

The reason I'd argue for using a collection.yml versus individual patterns is so that you don't have to open up each pattern to edit the front matter any time you want to adjust ordering.

I respectfully disagree with this reasoning, for a few reasons:

• It's no trouble to edit individual patterns because we'll likely be modifying those files with a far greater frequency than the collection info (which we'll likely set once and then rarely touch again).
• Consolidating all pattern ordering into a single file seems counter-intuitive when other preferences related to patterns (title, description, status, etc.) are all managed within the patterns themselves.
• Presumably the order information would need some sort of "key" to correspond to patterns. If that's the case, any changes to the pattern file (moving, renaming, deleting) will require that we manually manage collection.yml.
• We often have designers contributing patterns in parallel. Maintaining a single file corresponding to order feels like it will invite a greater frequency of merge conflicts, similar to those we see when multiple contributors make changes to dependencies in package.json.

I'm happy to be overruled if I'm in the minority. It just seems to me that preferences associated with individual patterns will be more portable if they're stored with the patterns themselves.

I think this can be done! Though again I'd almost argue for the collection.yml being the right place for this—the patterns are hidden vis-a-vis their collection, but not overall

I disagree, for the same reasons as above. 😬

(IIRC the requirement was to hide them from listings but still have them be available for partials and data?).

Not sure about data, but yes... hide from listings, but still available as partials. More info: #17

[erikjung]

@lyzadanger

On collection ordering, I prefer the discussed solutions in this order:

1. Specifying it in a single file (e.g. collection.yml)
2. Specifying it in each pattern's metadata
3. Specifying it by filename prefix

If we use a single file, how would the order be annotated? As an array of IDs?

I also should mention that consistency is more important to me than the chosen solution. I've never felt that the arbitrary ordering of things like patterns (which should have a pretty flat hierarchy IMO) is that important.

[tylersticka]

I've never felt that the arbitrary ordering of things like patterns (which should have a pretty flat hierarchy IMO) is that important.

I disagree. In our last big ol' billable project, the client had some pretty specific challenges directly related to patterns being ordered in a counter-intuitive (though alphabetical!) way. While that issue would have been partially alleviated by more granular pattern collections, I still think being able to prioritize patterns within a collection is really important.

[erikjung]

@lyzadanger @tylersticka Something to consider:

Could we handle ordering by specifying "dependencies" in the pattern front matter? For example:

---
# input-group.hbs
dependencies: [input, arrange]
---

---
# forms.hbs
dependencies: [input-group]
---

...and using their dependency graph to determine the ordering? Which, in the above example would be:

- arrange
- input
- input-group
- forms

[lyzadanger]

I like the idea of trying to keep as much collection-related metadata out of the individual pattern files. I've given one argument above (it seems easier to edit one file instead of each pattern in a collection), but this doesn't seem to be the case for Tyler.

I'd also add the technical argument that adding more and more "magical" properties to individual pattern file front matter introduces more likelihood for namespace collisions and makes traversal and object construction more challenging. I like the technical cleanliness of having all collection metadata in one place. order isn't about the pattern, it's about the collection it's in. To get even more forward-looking, I could conceive of a potential time in the future you guys might not even want a strict 1-to-1 relationship between patterns and collections (that is, potentially allowing a pattern to exist in more than one collection).

One of the things that has been hardest to understand and untangle in the origin code source is the distinction between patterns and collections. I'd like not to run into those pitfalls again. For the most part I've been acting the part of the "customer" to the design folks on the requirements for this project, but I have a slightly-stronger-than-average opinion on this one—I'd prefer to keep the collection metadata with the collection.

I want to be clear that ordering will be supported! The discussion here is just about implementation/methodology for that. I'm not abandoning the feature.

@erikjung:

Could we handle ordering by specifying "dependencies" in the pattern front matter?

I feel like I need to do a Phd to understand that 😄

[erikjung]

@lyzadanger

I feel like I need to do a Phd to understand that

var tree = {a: [], b: ['c', 'd'], c: ['d'], d: ['a']};
var resolved = DepResolver(tree);
console.log(resolved);//['a','d','c','b' ] 

https://www.npmjs.com/package/dependency-tree-resolver#examples

FWIW, this way we'd actually be specifying data that does relate to each pattern.

Just a 💭

[saralohr]

I see ups and downs for both ordering via front matter and ordering via yaml, but don't have a strong preference either way (I may after using it for a while, but alas, foresight often doesn't serve as well as hindsight).

Any subdirectory of the patterns directory with any immediate-children pattern files is considered a collection

Does this apply to ordering of immediate children of the patterns directory, ie ordering of collections and freestanding patterns, as well?

[lyzadanger]

Does this apply to ordering of immediate children of the patterns directory, ie ordering of collections and freestanding patterns, as well?

If I'm understanding the question correctly...any pattern files that are immediate children to the top-level patterns directory would be considered part of a collection—by default titled Patterns.

[lyzadanger]

I see ups and downs for both ordering via front matter and ordering via yaml

To be a touch pedantic, front matter is also YAML.

The drizzle builder is already happily capable of parsing JSON as well as YAML, so there's no reason the collection metadata file couldn't contain JSON (or the project could contain a mix, doesn't really matter as long as it's borne out by file extensions).

[tylersticka]

I feel like I need to do a Phd to understand that

Same. It's an interesting idea, @erikjung, but it feels to me like over-engineering... an opinion reinforced by your follow-up example. 😬

I'd also add the technical argument that adding more and more "magical" properties to individual pattern file front matter introduces more likelihood for namespace collisions and makes traversal and object construction more challenging.

Maybe we're misunderstanding what I'm after with pattern ordering, which may not be apparent based on our previous, filename-based solutions.

Let's take a hypothetical collection of button patterns:

base.hbs
disabled.hbs
favorite.hbs
rounded.hbs
sizes.hbs
social.hbs
variations.hbs

For the most part, it's fine that these are ordered alphabetically. But maybe I want to make extra sure that base is always first, variations and sizes are somewhere after that (since that information is more general), and maybe disabled should be at the end.

If I had an order property per pattern, that might look like this:

base.hbs        order: -2
sizes.hbs       order: -1
variations.hbs  order: -1
favorite.hbs
rounded.hbs
social.hbs
disabled.hbs    order:  1

The filename prefixes were always just a hack to get these sorts of optional, Flexbox-style order shifts.

I won't argue that storing order within individual patterns becomes silly if every pattern has a specific, intended order in the document flow. But looking through our past style guides, that usage is infrequent. The most common usage of order is to force a single pattern to be first in the listing... it's with that usage in mind that maintaining a separate list array feels like a heavy solution by comparison.

[gerardo-rodriguez]

I don't have much to add to the conversation, but @tylersticka's last message cleared up (for me) what this conversation is trying to solve/manage.

The filename prefixes were always just a hack to get these sorts of optional, Flexbox-style order shifts.

FWIW, I used this hack. 😅

[lyzadanger]

@tylersticka Your version:

base.hbs        order: -2
sizes.hbs       order: -1
variations.hbs  order: -1
favorite.hbs
rounded.hbs
social.hbs
disabled.hbs    order:  1

Would be a somewhat challenging algorithm to write. -1 relative to what? I'd have to look in every file inside of a collection and then assess what I find there. This leaves me scratching my head.

[erikjung]

@tylersticka

It's an interesting idea, @erikjung, but it feels to me like over-engineering... an opinion reinforced by your follow-up example. 😬

From the looks of it, defining dependencies is what we're already attempting to do with order, sans-semantics. Looking at your last example:

base.hbs        order: -2
sizes.hbs       order: -1
variations.hbs  order: -1
favorite.hbs
rounded.hbs
social.hbs
disabled.hbs    order:  1

Does this not reflect a sequence of dependencies?

Anyway, my two cents. I'll be fine with whatever we decide on.

[lyzadanger]

I realized I'm not doing a great service to anyone by splitting my attention between this and my book. Everyone always loses when I multitask too much :). I've been brainstorming a bit over lunch, but I'll return to this tomorrow with more focused attention.

[tylersticka]

Does this not reflect a sequence of dependencies?

It does, but that's not why I'm specifying order. I want the pattern higher up on the page so newcomers will have an easier time reading the section. In this case, saying "I want that because these patterns are dependent on the other" is accurate, but makes the feature less understandable.

Would be a somewhat challenging algorithm to write. -1 relative to what? I'd have to look in every file inside of a collection and then assess what I find there. This leaves me scratching my head.

Here's a small pen I whipped up to possibly help clarify my ideas: http://codepen.io/tylersticka/pen/BKdOYo?editors=0012

[erikjung]

It does, but that's not why I'm specifying order. I want the pattern higher up on the page so newcomers will have an easier time reading the section.

I've always understood (correctly or otherwise) the manual ordering of our pattern library as something we do to illustrate a linear progression from the "basics" to more complex, less foundational patterns that depend on those basics. Example: typography before components.

There are probably outliers, but I feel like most of the time we tend to order things in a way that coincidentally reflects their perceived dependencies (opposed to literal: CSS source order).

That's the only reason I thought I'd toss in the idea of using "dependency" annotations to achieve sorting. I think the example of using a relative order value is fine too. It makes sense. Here's what I had in mind with the other suggestion: https://tonicdev.com/erikjung/56fbd53696866f11009caeef

[lyzadanger]

Someone should start counting the agendas in this thread 😆

I'm not innocent, either!

[tylersticka]

@erikjung That proof of concept is helpful to see, but it also confirms my assumption that this would be more difficult to use day to day. It's just way more verbose to achieve the same thing, y'know? Instead of specifying a single integer value in the three patterns that should be ordered differently, you're specifying arrays of one or more keys in all but one of the patterns.

(That tonic output is sexy, though!)

Someone should start counting the agendas in this thread 😆

I'm not sure what this means. 😕

I just want the thing I use every day to be easy to use. 🦄

[lyzadanger]

I'm not sure what this means. 😕

I was just joking. It's fun to see so many different opinions and approaches! Agenda was a dumb word to use. Ideas may have been better.

[lyzadanger]

Here's the direction this has gone...I feel like this is a solid solution; hope it doesn't infuriate anyone too badly...

One can add an optional file to any pattern-containing directory. That file is called collection.+(yml|yaml|json). That file may contain some metadata about a collection, including, so far:

• order: An array of patterns in the order you'd like to have them rendered on the pattern collection page.
• hidden: An array of any patterns in the collection you'd like to hide.
• name: Pattern collection name (default is based on directory name).

order and hidden array elements are the 'keys" of patterns—just their filenames without extensions.

If an order array is provided but there are any patterns unaccounted for, those patterns will be appended to the end of the list in their default (filesystem) order.

Hidden patterns are parsed and available for other parts of the build, but are not rendered on the pattern collection page.

An example collection.yml file:

name: These are Cool
order:
  - blissful
  - awareness
  - depredation
  - salmonella
  - frenetic
hidden:
  - albumen

[tylersticka]

I guess I've been overruled.

Maybe in the future I'll write a PR to make stuff work the way I'd like it to. I stand by my previous arguments.

[lyzadanger]

Maybe we could try to find a compromise? Right now I'm trying to get this done, and implementing it on a per-pattern basis would be a more challenging algorithm to write, and would cause more potential for namespace collisions. I can see if I can find a way to do it the Tyler way this week.

[lyzadanger]

I think this can be done now.