Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Data folder #85

Merged
merged 34 commits into from Sep 11, 2019
Merged

Data folder #85

merged 34 commits into from Sep 11, 2019

Conversation

sc0ttj
Copy link
Owner

@sc0ttj sc0ttj commented Aug 21, 2019

An early attempt at solving #65

What's the improvement?

New feature: the assets/data folder

Site-wide data: Put .yml, .csv, or .sh files in the assets/data/ folder, and the data they contain will be available to all templates at build time.

Page specific data: Put .yml, .csv, or .sh files in the assets/data/my-cool-page/ folder, and the data they contain will be available to the page my-cool-page only at build time.

Details

  • less code, more config
    • site data now defined in YAML (assets/data/site.yml)
    • page/post data defined in YAML in frontmatter (in top of .mdsh files)
  • better separation of build data and templates
    • only get site data from assets/data folder (not .site_config)
    • only get post/page data from yaml frontmatter or assets/data
    • fewer custom functions used by templates to generate data, more arrays
  • nicer file structure
    • fewer files, simpler file naming/layout
    • changed various scripts into functions
  • cleaner templates
    • removed all shell funcs in templates, replaced with:
      • site/page arrays accessed via the foreach iterator
      • better partial handling

Other new features

The foreach iterator

  • adds a foreach iterator to the templating system

Easier data debugging

  • added a DEBUG_DATA env var, set to true to see site data printed in terminal during build

page_layout variable

Easily render pages with different layouts

Define it in the page front matter, or pass it in via CLI when running create_page.sh (defaults to main if empty, and must be the name of a template file in .app/templates/*/)..

Fixes

  • added .nojekyll file, used by GitHub to help identify Jekyll sites (or not)
    ebc78c3

Implementation of data folder

Added example data files:

site.yml, products.yml, people.csv, arrays-example.sh

Pull in data at build time:

Added a .yml parser & csv parser func, updated cli-interface to parse data files on each build:

Updates in .app.cli-interface.bash to scan the assets/data dir for files, and parse them into vars and arrays that our build scripts and templates can use...

YAML parsers used:

  • .app/functions/yay - custom version of yay with added stuff:
    • fixes and added support for multiple nested levels
    • fixes for handling simple lists, and other YAML syntaxes
    • TLDR: better at creating the associative arrays our templates need
  • .app/functions/ysh - a parser called yaml.sh
    • used only inside the lookup method
    • the lookup method only calls ysh if no site data vars/arrays found
    • else it returns the matching values from the vars/arrays

CSV parser:

  • shell function called csvToData, in .app/functions/liquid_filters.bash
  • turns CSV data into bash associative arrays that our templates can use

Much easier to use arrays in templates:

Added a foreach iterator function, to be used in the templates, small fix to mo script to make it work OK.

An example!!

  1. assets/data/products.yml contains:
  product1:
    name: Foo
    price: 10
  product2:
    name: Bar
    price: 20
  1. so you access it in your templates like so:
  {{#foreach product in products}}
    {{product.name}} costs {{product.price}}
  {{/foreach}}

TODO:

  • test foreach iterator with mutiple filters
  • test page specifc data

- New folder: assets/data

  put .yml or .sh files in the folder,
  and the data they contain will be
  available to the templates at build
  time

- Added example data files:

  site.yml, products.yml, and
  arrays-example.sh

- Pull in data at build time:

  added a .yml parser, updated cli-
  interface, gets data on each build

- Much easier to use arrays in templates:

  added a foreach iterator function, to
  be used in the templates, small fix to
  mo script to make it work ok

- Now an example!!

  1. `assets/data/products.yml` contains:

  product1:
    name: Foo
    price: 10
  product2:
    name: Bar
    price: 20

  2. so you access it in your templates like so:

  {{#foreach product in products}}
    {{product.name}} costs {{product.price}}
  {{/foreach}}

NOTE: still a LOT of refactoring to do in order
      to make use of data folder feature properly
Can now handle simple lists (dashed lists), and creates
variables names of every entry
- added support for dash notation lists
- added more _ concatenated vars to output
@sc0ttj sc0ttj mentioned this pull request Aug 21, 2019
- the CSV files must have headers, be comma separated
- (added new `csv_to_data` function, also works as liquid filter)
- plus some fixes: renamed `basename` and `dirname` filters to `base_name` and `dir_name`
- added example CSV file
This will now work in templates:

```
{{lookup products.product1.name}}
```

The `lookup` method will return the values of the given
data object (either from existing array, var or from .yml
files (using the new `ysh` script to do the "query")
This makes it easier to access the data generated from any
CSV files in the assets/data folder using `lookup`
…a` method, to debug the data created (not finished..)
- less code, more config
    - posts data now comes from YAML (in frontmatter of `.mdsh files)
    - site data now also comes from YAML
- better separation of build data and templates
    - only get site data from `assets/data` folder (not `.site_config`)
    - only get post/page data from yaml frontmatter or `assets/data`
    - fewer custom functions used by templates to generate data, more arrays
- nicer file structure
    - fewer files, simpler file naming/layout
    - changed various scripts into functions
- nicer build data
    - better variable naming: use only `$site_` and `$page_`, removed `$blog_` & `$post_`
    - more `site_*` arrays available
    - more `page_*` data available
- cleaner templates
    - removed all shell funcs in templates, replaced with:
        - site/page arrays accessed via the `foreach` iterator
        - better partial handling
- new features
    - added a `DEBUG_DATA` env var, set to `true` to see site data printed in terminal during build
    - (possibly) fixed `foreach` iterator to work with liquid filters (see `_site_navigations.mustache`)
    - added .nojekyll file, used by GitHub to help identify Jekyll sites (or not)
…time:

This enables putting data files inside folders named after your pages, to
include data only for that page. Example: For post "My post", which gets
the slug "my-post", create the folder "assets/data/my-post/" and put .sh,
.yml or .csv files in there.
- rename "next_post" hash to "page_next"
- rename "prev_post" hash to "page_prev"

- this namespaces the data more consistently with
  existing page data
about "page_layout":
  - should match an existing template name (.app/templates/*/<name>.mustache)
  - can be set in frontmatter of post, or pass in via CLI
  - `create_page.sh` now calls `render $page_layout` to render pages
  - page_layout falls back to "main" if empty
- add "permalink: some/name" to your .mdsh front matter
  and it will be created as a symlink at build time.

- old permalinks are automatically removed if you change
  the value
@sc0ttj sc0ttj mentioned this pull request Sep 11, 2019
@sc0ttj sc0ttj merged commit 318492a into gh-pages Sep 11, 2019
@sc0ttj sc0ttj deleted the data-folder branch September 11, 2019 21:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant