Allow reading data files bundled within theme-gems

docs/_docs/themes.md

@@ -288,6 +288,75 @@ Your theme's styles can be included in the user's stylesheet using the `@import`
 ```
 {% endraw %}
 
+### Data Files {%- include docs_version_badge.html version="4.1.0" -%}
+
+Data files located inside the directory `_data` at the root of the theme will be read and merged into your site's internal
+data hash unless the internal data hash already contains a `key` of the same name. This allows theme-authors to easily
+define certain elements like the structure of various types of site-navigation, or ship pre-defined localization strings
+with their gem.
+
+Like other resources bundled in the theme, data read-in the theme-gem can be overridden by having a similar data file at
+the user's data-file directory at source.
+
+Theme authors may either ship data in a single file or organize them in sub-directories. The two cases below are equally
+valid:
+
+#### Case A
+
+```yaml
+# <theme.root>/_data/navigation/top_nav.yml
+
+- title  : "About Me"
+  url    : "/about/"
+- title  : "Contact"
+  url    : "/contact/"
+```
+
+```yaml
+# <theme.root>/_data/navigation/side_nav.yml
+
+- title  : "Link A"
+  url    : "/link-a.html"
+- title  : "Link B"
+  url    : "/link-b.html"
+```
+
+#### Case B
+
+```yaml
+# <theme.root>/_data/navigation.yml
+
+top_nav:
+  - title  : "About Me"
+    url    : "/about/"
+  - title  : "Contact"
+    url    : "/contact/"
+
+side_nav:
+  - title  : "Link A"
+    url    : "/link-a.html"
+  - title  : "Link B"
+    url    : "/link-b.html"
+```
+
+Similarly, user-overrides may be a single file as well. Do note, that if a mapping key has been declared, it will override
+the entire block within theme data even if the sub-keys are lesser in number. In the example below, the resulting `side_nav`
+map will contain just *one key/value* pair even if the theme shipped with two pairs:
+
+```yaml
+#<site.root>/_data/overrides.yml
+
+top_nav:
+  - title  : "About Jekyll"
+    url    : "/about/"
+  - title  : "Jekyll at Github"
+    url    : "/community/"
+
+side_nav:
+  - title  : "News"
+    url    : "/news/"
+```
+
 ### Theme-gem dependencies {%- include docs_version_badge.html version="3.5.0" -%}
 
 Jekyll will automatically require all whitelisted `runtime_dependencies` of your theme-gem even if they're not explicitly included under the `plugins` array in the site's config file. (Note: whitelisting is only required when building or serving with the `--safe` option.)

features/theme.feature

@@ -14,6 +14,22 @@ Feature: Writing themes
     And the my-cool-theme directory should exist
     And the "my-cool-theme/CODE_OF_CONDUCT.md" file should exist
 
+  Scenario: A theme with data files
+    Given I have a configuration file with "theme" set to "test-theme"
+    And I have a "index.html" file with content:
+    """
+    ---
+    ---
+    {% assign menu = site.data.navigation %}
+    {% for nav in menu.topnav %}
+      <a href="{{ nav.url | relative_url }}">{{ nav.title | escape }}</a>
+    {% endfor %}
+    """
+    When I run jekyll build
+    Then I should get a zero exit status
+    And the _site directory should exist
+    And I should see "\n\n  <a href=\"/about/\">About</a>\n\n  <a href=\"/contact/\">Contact</a>\n" in "_site/index.html"
+
   Scenario: A theme with SCSS
     Given I have a configuration file with "theme" set to "test-theme"
     When I run jekyll build

features/theme_data.feature

@@ -0,0 +1,77 @@
+Feature: Theme data
+  As a hacker who likes to share my expertise
+  I want to be able to embed data into my gemified theme
+  In order to make the theme slightly dynamic
+
+  Scenario: A theme with data files
+    Given I have a configuration file with "theme" set to "test-theme"
+    And I have a "index.html" file with content:
+    """
+    ---
+    ---
+    {% assign menu = site.data.navigation %}
+    {% for nav in menu.topnav %}
+      <a href="{{ nav.url | relative_url }}">{{ nav.title | escape }}</a>
+    {% endfor %}
+    """
+    When I run jekyll build
+    Then I should get a zero exit status
+    And the _site directory should exist
+    And I should see "\n\n  <a href=\"/about/\">About</a>\n\n  <a href=\"/contact/\">Contact</a>\n" in "_site/index.html"
+
+  Scenario: A site has other data files
+    Given I have a configuration file with "theme" set to "test-theme"
+    And I have a _data directory
+    And I have a "_data/reviews.yml" file with content:
+    """
+    docs:
+    - title  : The Iron Throne
+      url    : /iron-throne/
+    - title  : Fire & Ice
+      url    : /fire-n-ice/
+    """
+    And I have a "index.html" file with content:
+    """
+    ---
+    ---
+    {% assign mainmenu = site.data.navigation %}
+    {% for nav in mainmenu.topnav %}
+      <a href="{{ nav.url | relative_url }}">{{ nav.title | escape }}</a>
+    {% endfor %}
+
+    {% assign reviews = site.data.reviews %}
+    {% for item in reviews.docs %}
+      <a href="{{ item.url | relative_url }}">{{ item.title | escape }}</a>
+    {% endfor %}
+    """
+    When I run jekyll build
+    Then I should get a zero exit status
+    And the _site directory should exist
+    And I should see "\n\n  <a href=\"/about/\">About</a>\n\n  <a href=\"/contact/\">Contact</a>\n\n\n\n\n  <a href=\"/iron-throne/\">The Iron Throne</a>\n\n  <a href=\"/fire-n-ice/\">Fire &amp; Ice</a>\n" in "_site/index.html"
+
+  Scenario: A site has a data file to override theme data
+    Given I have a configuration file with "theme" set to "test-theme"
+    And I have a _data directory
+    And I have a "_data/navigation.yml" file with content:
+    """
+    topnav:
+    - title  : About Krypton
+      url    : /about/
+    - title  : Smallville Diaries
+      url    : /smallville/
+    - title  : Contact Kal-El
+      url    : /contact-us/
+    """
+    And I have a "index.html" file with content:
+    """
+    ---
+    ---
+    {% assign mainmenu = site.data.navigation %}
+    {% for nav in mainmenu.topnav %}
+      <a href="{{ nav.url | relative_url }}">{{ nav.title | escape }}</a>
+    {% endfor %}
+    """
+    When I run jekyll build
+    Then I should get a zero exit status
+    And the _site directory should exist
+    And I should see "\n\n  <a href=\"/about/\">About Krypton</a>\n\n  <a href=\"/smallville/\">Smallville Diaries</a>\n\n  <a href=\"/contact-us/\">Contact Kal-El</a>\n" in "_site/index.html"

lib/jekyll/reader.rb

@@ -16,7 +16,7 @@ def read
       read_directories
       read_included_excludes
       sort_files!
-      @site.data = DataReader.new(site).read(site.config["data_dir"])
+      read_site_data
       CollectionReader.new(site).read
       ThemeAssetsReader.new(site).read
     end
@@ -183,5 +183,21 @@ def read_included_file(entry_path)
         site.static_files.concat(StaticFileReader.new(site, dir).read(file))
       end
     end
+
+    # Read data files within current site.
+    #
+    # If the site uses a theme and the theme has a `_data` directory at its root, read the files
+    # within and passively add them to the site's data hash.
+    # Only those keys that do not already exist in the site's data hash will be inserted from
+    # the theme-gem's data files.
+    #
+    # Returns a hash appended with new data
+    def read_site_data
+      @site.data = DataReader.new(site).read(site.config["data_dir"])
+      return unless (data_path = site.theme&.data_path)
+
+      theme_data = DataReader.new(site, "theme").read(data_path)
+      @site.data = Utils.deep_merge_hashes(theme_data, @site.data)
+    end
   end
 end

lib/jekyll/readers/data_reader.rb

@@ -3,10 +3,12 @@
 module Jekyll
   class DataReader
     attr_reader :site, :content
-    def initialize(site)
+
+    def initialize(site, mode = "source")
       @site = site
       @content = {}
       @entry_filter = EntryFilter.new(site)
+      @mode = mode
     end
 
     # Read all the files in <dir> and adds them to @content
@@ -16,7 +18,7 @@ def initialize(site)
     # Returns @content, a Hash of the .yaml, .yml,
     # .json, and .csv files in the base directory
     def read(dir)
-      base = site.in_source_dir(dir)
+      base = compute_absolute_path(dir)
       read_data_to(base, @content)
       @content
     end
@@ -36,7 +38,7 @@ def read_data_to(dir, data)
       end
 
       entries.each do |entry|
-        path = @site.in_source_dir(dir, entry)
+        path = compute_absolute_path(dir, entry)
         next if @entry_filter.symlink?(path)
 
         if File.directory?(path)
@@ -71,5 +73,11 @@ def sanitize_filename(name)
       name.gsub(%r![^\w\s-]+|(?<=^|\b\s)\s+(?=$|\s?\b)!, "")
         .gsub(%r!\s+!, "_")
     end
+
+    private
+
+    def compute_absolute_path(*entries)
+      @mode == "theme" ? @site.in_theme_dir(*entries) : @site.in_source_dir(*entries)
+    end
   end
 end

lib/jekyll/theme.rb

@@ -33,6 +33,10 @@ def sass_path
       @sass_path ||= path_for "_sass"
     end
 
+    def data_path
+      @data_path ||= path_for "_data"
+    end
+
     def assets_path
       @assets_path ||= path_for "assets"
     end

test/fixtures/test-theme/_data/navigation/sidenav.yml

@@ -0,0 +1,4 @@
+- title  : Link Item A
+  url    : /item-a.html
+- title  : Link Item B
+  url    : /item-b.html

test/fixtures/test-theme/_data/navigation/topnav.yml

@@ -0,0 +1,4 @@
+- title  : About
+  url    : /about/
+- title  : Contact
+  url    : /contact/

test/source/_data/navigation/sidenav.yml

@@ -0,0 +1,6 @@
+- title  : Blog
+  url    : /blog/
+- title  : News
+  url    : /news/
+- title  : Register
+  url    : /register.html

test/source/theme_data_test/output.html

@@ -0,0 +1,17 @@
+
+<div id="top_nav">
+
+  <a href="/about/">About</a>
+
+  <a href="/contact/">Contact</a>
+
+</div>
+<div id="side_nav">
+
+  <a href="/blog/">Blog</a>
+
+  <a href="/news/">News</a>
+
+  <a href="/register.html">Register</a>
+
+</div>

test/source/theme_data_test/override_output.html

@@ -0,0 +1,17 @@
+
+<div id="top_nav">
+
+  <a href="/about/">About</a>
+
+  <a href="/contact/">Contact</a>
+
+</div>
+<div id="side_nav">
+
+  <a href="/blog/">Blog</a>
+
+  <a href="/news/">News</a>
+
+  <a href="/register.html">Register</a>
+
+</div>

test/source/theme_data_test/source.html

@@ -0,0 +1,15 @@
+---
+permalink: /theme_data_test/nav.html
+---
+
+{% assign menu = site.data.navigation %}
+<div id="top_nav">
+{% for nav in menu.topnav %}
+  <a href="{{ nav.url | relative_url }}">{{ nav.title | escape }}</a>
+{% endfor %}
+</div>
+<div id="side_nav">
+{% for nav in menu.sidenav %}
+  <a href="{{ nav.url | relative_url }}">{{ nav.title | escape }}</a>
+{% endfor %}
+</div>

test/source/theme_data_test/source_override.html

@@ -0,0 +1,15 @@
+---
+permalink: /theme_data_test/nav-override.html
+---
+
+{% assign menu = site.data.navigation %}
+<div id="top_nav">
+{% for nav in menu.topnav %}
+  <a href="{{ nav.url | relative_url }}">{{ nav.title | escape }}</a>
+{% endfor %}
+</div>
+<div id="side_nav">
+{% for nav in menu.sidenav %}
+  <a href="{{ nav.url | relative_url }}">{{ nav.title | escape }}</a>
+{% endfor %}
+</div>

test/test_data_reader.rb

@@ -14,4 +14,42 @@ class TestDataReader < JekyllUnitTest
       )
     end
   end
+
+  context "Theme with data files in a site" do
+    setup do
+      @site = fixture_site(
+        "theme" => "test-theme"
+      )
+      assert @site.theme.data_path
+      @theme_data = DataReader.new(@site, "theme").read("_data")
+      @site.process
+    end
+
+    context "without data files at source" do
+      should "read data files in theme gem" do
+        assert_equal @theme_data["navigation"]["topnav"], @site.data["navigation"]["topnav"]
+      end
+
+      should "use data from theme gem" do
+        assert_equal(
+          File.read(source_dir("theme_data_test", "output.html")),
+          File.read(@site.in_dest_dir("theme_data_test/nav.html"))
+        )
+      end
+    end
+
+    context "with same data filenames at source" do
+      should "override theme data" do
+        file_content = SafeYAML.load_file(source_dir("_data/navigation/", "sidenav.yml"))
+        assert_equal file_content, @site.data["navigation"]["sidenav"]
+      end
+
+      should "also use data from theme gem" do
+        assert_equal(
+          File.read(source_dir("theme_data_test", "override_output.html")),
+          File.read(@site.in_dest_dir("theme_data_test/nav-override.html"))
+        )
+      end
+    end
+  end
 end