Add new flag to Google Sheets processor to treat content as a key/value store

[jeremydw]

To make it easier to map content from a Google Sheet to a page's structure, allow the developer to indicate that the Google Sheet should be processed as a key/value store instead of a list of rows.

For example, given the following Google Sheet:

key, value
key1, value1
key2, value2

WITH the new flag:

# file.yaml
key1: value1
key2: value2

WITHOUT the new flag:

# file.yaml
- key: key1
  value: value1
- key: key2
  value: value2

We'd want similar behavior to the SO answer here: http://stackoverflow.com/a/16641671/916643

[stevenle]

Any particular thoughts on how localization overrides might work with something like this?

[jeremydw]

As part of the preprocessor config we could add a locale option. That would allow folks to use different sheets to manage localized data in the same YAML file.

These are the new config options I added in the previous commits:

preserve: builtins  # Prevents fields starting with $ from being overwritten.
preserve: all  # Preserves all fields in the existing YAML file.
preserve: null  # Overwrites the entire yaml file (default, current behavior).
format: map  # Formats the CSV as a mapping of keys to values.
format: list  # Formats the CSV as a list of rows (default, current behavior).

And we could add locale to the above to specify which locale to save the data to. Presently this would really only have meaning for CSVs saved to YAML files. Thoughts?

[stevenle]

Just to make sure I understand, so you'd have the first, default sheet with something like:

key, value
foo, bar

And in another sheet for a particular locale, you'd have something like:

key, value
foo, baz

If I specify something like:

locale: <sheet id>

Would it automatically output the data into a separate yaml file with the overrides? Is that the idea?

[stevenle]

Oh, I guess it would be in a separate front matter. So in that example, the yaml would be outputted as:

# content.yaml

foo: bar
---
$locale: <locale>
foo: baz

Is that right?

[jeremydw]

It would require two preprocessors (one for each worksheet). If the preprocessors were configured to save to the same file, and if locale was specified, you could have this:

# podspec.yaml
preprocessors:
- kind: google_sheets
  path: /content/pages/home.yaml
  id: <sheet id>
- kind: google_sheets
  path: /content/pages/home.yaml
  id: <sheet id>
  locale: fr_FR

# /content/pages/home.yaml
---
key: value
foo: bar
---
$locale: fr_FR
key: value
foo: baz

Ultimately we could add a new way of specifying data sources directly in the YAML files but that's a bigger change that would require some good thought. For now the preprocessors build on what we already have and it would work well.

[stevenle]

SGTM

[jeremydw]

I noticed that this also really benefits from the --tag option that you added. So you can add tags: [home] to these pages and preprocess all the "home" content with just grow preprocess --tag=home. Works well!

[jeremydw]

Implemented and documented at https://grow.io/docs/preprocessors/#google-sheets

I'm going to close this issue now, and I don't think we should support any special behavior for multi-part localized documents with this preprocessor. Instead, if you would like to bring in localized content from a Google Sheet, I think it should be done using the multi-file localization feature that was implemented in #229.

An example configuration for that:

preprocessors:
- kind: google_sheets
  path: /content/pages/page.yaml
  id: 1ZhJshmT2pZq_IALA6leXJ0oRuKVO12N9BbjAarAT8kI
  gid: 0  # gid corresponding to base locale worksheet.
  format: map
- kind: google_sheets
  path: /content/pages/page@fr.yaml
  id: 1ZhJshmT2pZq_IALA6leXJ0oRuKVO12N9BbjAarAT8kI
  gid: 1  # gid corresponding to "fr" worksheet.
  format: map