Single source of truth distributes shared config between layers, allowing extension

[davidcmoulton]

Purpose

This PR addresses two orthogonal requirements for configuration management in the pattern library:

1. A single source of truth for configuration that's to be shared between technology layers, for example media query breakpoints are always needed in the styling layer, but are sometimes also needed in the behavioural layer.

2. The ability of a user of the system to modify the supplied configuration in order to override the provided defaults, for example changing the colours, fonts or breakpoints, or to augment the existing default configuration with additional configuration, or to remove the supplied default configuration completely to replace it with their own.

Misc notes

• The generated sass & json files are currently committed. They'll probably be removed later once this has proven stable as they're derived files. Removed.
• This approach reverses the earlier thinking on namespacing different implementors' property names. For example initially I was imagining a default of color: $libero-color-text-normal that could be changed to color: $thirdparty-color-text-normal by a third party. The approach in this PR provides a different approach: instead color: $color-text-normal will always be how the color is set, but the value defined by $color-text-normal can be changed using a custom config file.

The rest of this description describes how what's in the PR works for a user trying to implement the config system. Partly so I don't forget how it works(!), and partly because it could be used as the basis for docs on how to configure the Libero pattern library when we're into that phase.

---

If you don't want to alter the default Libero config, and no knowledge is shared across technology boundaries, then there's no need to use this config system. Otherwise, the following applies.

1. Sharing configuration across technology boundaries

The aim is to avoid having to maintain duplicate knowledge across technology boundaries: for example, we want to be able to define breakpoints in one place, and have that knowledge persist to the correct technology layer without having to maintain it in multiple places.

For configuration that needs to be shared across two or more of SASS, JavaScript and the templating layer, the base configuration in /libero-config/config--libero-default.js defines the base config object as

config = { data: {}, layerAllocations: {} };

Definingconfig.data

Each top level property of data defines a slice of configuration that should be distributed to more than one technology layer. For example the breakpoints in/libero-config/config--libero-default.js are defined by:

config.data.breakpoints = {site: {}};
config.data.breakpoints.site.x_small = 320;
config.data.breakpoints.site.small = 480;
...

(Note that these top level properties of data can be called anything you like.)

Values may be defined as simple expressions using the form !expression [my-expression]. For example,

config.data.baselinegrid.space.extra_small_in_px = 12;
config.data.baselinegrid.space.small_in_px = '!expression baselinegrid.space.extra_small_in_px * 2';

will cause the final value of small_in_px to be twice that of extra_small_in_px. By using jexl under the bonnet, this allows a relationship to be established that can be maintained even if the original predicate is changed. So if, for example, someone wanted to change the value of extra_small_in_px, but to still ensure that the value of small_in_px remained twice its new size, they could perform this override in a custom config file (see the sction below: "Extending or replacing default Libero config"), but they'd only need to redefine the value they actually wanted to change, the !expression would ensure that the derived values stayed true to the new base value.

Allocating config data to technology layers with config.layerAllocations

layerAllocations defines which config.data properties each technology layer will receive, for example:

config.layerAllocations = {
  sass: ['breakpoints'],
  js: ['breakpoints']
};

determines that both the SASS and JavaScript layers will receive the configuration defining the breakpoints.

Note that it's not yet clear what form we might want the configuration for the template layer to take as we're not yet sure how we might want to use it. Although it's supported to allocate a value to a layerAllocations.templates array, it's not currently used.

Distribution to SASS

config.data properties specified in config.layerAllocations.sass are used to populate their respective sass variables files. For example, the config

const config = { data: {}, layerAllocations: {} };
config.data.breakpoints = {site: {}};
config.data.breakpoints.site.x_small = 320;
config.data.breakpoints.site.small = 480;

config.layerAllocations.sass = ['breakpoints'];

would cause the config.data.breakpoints property to be written as a sass variable file called _variables--breakpoints.scss, with this content:

$breakpoints-site-x_small: 320;
$breakpoints-site-small: 480;

Distribution to JavaScript

All config.data properties specified in config.layerAllocations.js are incorporated into a single json file called configForJs.json. For example

const config = { data: {}, layerAllocations: {} };
config.data.breakpoints = {site: {}};
config.data.breakpoints.site.x_small = 320;
config.data.breakpoints.site.small = 480;

config.layerAllocations.js = ['breakpoints'];

would cause theconfig.data.breakpoints property to be written to configForJs.json, the file used for handling config in the JavaScript layer:

{
  ...
  "breakpoints":{"site":{"x_small":320,"small":480}}
  ...
}

Distribution to templates

[Not yet implemented.]

2. Extending or replacing default Libero config

The default Libero configuration is defined in /libero-config/config--libero-default.js. This should be considered read only by implementors. Its use may be omitted altogether, but if it remains in use it should not be altered.

Adjustment to the config should be provided by an additional file or set of files. Add custom config files to /libero-config/. These files may provide any desired changes to existing properties, and add in new properties, and add layerAllocations as desired. An example custom config file, /libero-config/config--custom.js is provided as an example.

The custom config files may be named anything. In order to use them, they must be added to the configPaths array in /libero-config/bin/distributeConfig.js. (This may be extracted in a later feature so it can be set with environment variables or similar.) The order the files are added to the configFiles array is important for handing conflicts: when two identical properties are defined in more than one config file, the conflicting property in the latest config file in the list will overwrite the conflicting property in earlier ones.

[davidcmoulton]

Update: Merging in #15 has fixed this, thanks @thewilkybarkid.

There looks to be something like a race condition around deleting files when running the gulp task incorporateSharedConfig within the default pipeline. Will try replacing del with gulp-clean to see if that makes a difference.

[thewilkybarkid]

Sounds like #15.

[stephenwf]

We've had a similar requirement on a different project, I think this approach makes sense for your use case and I wouldn't change it, but I thought I'd share the approach we took.

So for our use case we wanted to share some simple colour accents for a UI between JS and Sass, now we wanted the variables to be extracted as native sass elements (not as strings!) so we found a library that will convert many types into Sass compatible: https://www.npmjs.com/package/json-sass

This didn't really help with actually importing them though, or sharing them. What we found is that node-sass can accept both a sassOptions.filePath (as used internally by gulp-sass etc) but also sassOptions.data and it will use both. So essentially, sassOptions.data could be some inline sass that we can inject at built time (like shared variables).

In the end we published this tiny tool: https://www.npmjs.com/package/sass-define that lets you define Sass variables for use in that file. e.g.:

const sassDefine = require('sass-define');

const sassOptions = {
  data: sassDefine({
    backgroundColour: '#ff0000'
  }),
  // or
  // data: sassDefine(require('./path/to/shared/vars.js}'))
};

and pass them to webpack sass loader options in our case. The output of sassDefine would be something along the lines of:

$backgroundColour: #ff0000;

Where unlike json-sass it loops through each key of the configuration and creates a sass variable. But it supports objects, arrays, numbers, and all sorts of units too.

As for our stylesheets, since theres no visible sass, we set the variables that could be customised with the !default keyword. So our base configuration looks like:

$backgroundColour: #00ff00 !default;

and we still got all the autocomplete good-ness in our IDEs.

The upside of the approach you have here is there is no need for this default! behaviour. Our use-case was primarily around shipping sass in a package with themeX and using it in another project with themeY. Just wanted to share the approach!

[thewilkybarkid]

Probably don't actually want to support this? We know that we want to support extensions of the pattern library, but forking isn't really the way.

[davidcmoulton]

Something to think about. Not sure just at the moment how else we might handle extensions. Will ponder.

[davidcmoulton]

Will remove documentation, but leave implementation in for now, undocumented & so subject to change.

[thewilkybarkid]

Pretty wordy, /source/js/config.json?

[davidcmoulton]

Fair point. I like/source/js/derivedConfig.json: it indicates that the file shouldn't be directly edited, and we can't insert that information as a comment into the file itself as it's json.

[thewilkybarkid]

Should be generating all the 'variables' files? If so, /source/css/sass/_variables--[propertyname].sass or /source/css/sass/variables/[propertyname].sass?

(Also, sass inside css?)

[davidcmoulton]

Originally I though that it'd only need to generate those that are either extended by additional config, or that contain knowledge shared between technology layers, but now I'm thinking that it should generate all of them.

[davidcmoulton]

sass inside css because of how it's used during the export process. I'm not wedded to that structure, but a change is outside the scope of this PR.

[thewilkybarkid]

Can a sub-process be avoided?

[davidcmoulton]

Possibly. I originally tried to integrate it into the gulp process more effectively, but it didn't work. I think it's time for another go though.

[thewilkybarkid]

Indentation's a bit hard to read.

[davidcmoulton]

Yeah, I'll fix that.