Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP

Loading…

Added config includes to solve #514 and #703 #722

Closed
wants to merge 2 commits into from

2 participants

@xenji

including the corresponding tests and fixtures.

You can now use a YAML list to add configuration files to be included
in the order of the list. The main configuration is included at the
final step to ensure that every value can be overridden.

The syntax supports globs (to be used with Dir[pattern]), relative
paths to the config value of source and absolute paths.

Using this sample config:

additional_configs:
  - additional_configs/_config_a.yml
  - additional_configs/_config_b.yml

or the glob version:

additional_configs:
  - additional_configs/_config_*.yml

the new deep merge order is in both cases:
Jekyll::DEFAULTS < _config_a < _config_b < _config

Using globs means to rely on the sorting that Dir[pattern] resolves.

I've added additional tests for three use cases:

  • Having one inclusion
  • Having a glob inclusion
  • Having one inclusion with override in the main config
@xenji xenji Added config includes to solve #514 and #703
including the corresponding tests and fixtures.

You can now use a YAML list to add configuration files to be included
in the order of the list. The main configuration is included at the
final step to ensure that every value can be overridden.

The syntax supports globs (to be used with `Dir[pattern]`), relative
paths to the config value of `source` and absolute paths.

Using this sample config:
```
additional_configs:
  - additional_configs/_config_a.yml
  - additional_configs/_config_b.yml
```
or the glob version:
```
additional_configs:
  - additional_configs/_config_*.yml
```
the new deep merge order is in both cases:
`Jekyll::DEFAULTS < _config_a < _config_b < _config`

Using globs means to rely on the sorting that Dir[pattern] resolves.

I've added additional tests for three use cases:
* Having one inclusion
* Having a glob inclusion
* Having one inclusion with override in the main config
2961201
@xenji

@StevenBlack As written above, using globs means to rely on the sorting that Dir[pattern] resolves. This means in general, that your OS will take over the sorting and give it back to the ruby interpreter. As far as I know, you should not see any differences between file systems, but I won't bet on it.

I think a sorting like

10-less-important.yml
20-more-important.yml
99-most-important.yml

might be suitable to ensure an order on most file systems. If you need any of the configs at first and cannot change it's name, you might name it explicitly before the wildcard include. The patch checks for already included files and won't do it twice.

additional_configs:
  - additional_configs/_config_very_important.yml
  - additional_configs/_config_*.yml

HTH?

@parkr
Owner

We're going to go with a simpler solution via the CLI.

@parkr parkr closed this
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Commits on Dec 31, 2012
  1. @xenji

    Added config includes to solve #514 and #703

    xenji authored
    including the corresponding tests and fixtures.
    
    You can now use a YAML list to add configuration files to be included
    in the order of the list. The main configuration is included at the
    final step to ensure that every value can be overridden.
    
    The syntax supports globs (to be used with `Dir[pattern]`), relative
    paths to the config value of `source` and absolute paths.
    
    Using this sample config:
    ```
    additional_configs:
      - additional_configs/_config_a.yml
      - additional_configs/_config_b.yml
    ```
    or the glob version:
    ```
    additional_configs:
      - additional_configs/_config_*.yml
    ```
    the new deep merge order is in both cases:
    `Jekyll::DEFAULTS < _config_a < _config_b < _config`
    
    Using globs means to rely on the sorting that Dir[pattern] resolves.
    
    I've added additional tests for three use cases:
    * Having one inclusion
    * Having a glob inclusion
    * Having one inclusion with override in the main config
Commits on Jan 1, 2013
  1. @xenji
This page is out of date. Refresh to see the latest.
View
85 lib/jekyll.rb
@@ -27,6 +27,7 @@ def require_all(path)
require 'pygments'
# internal requires
+require 'jekyll/configuration'
require 'jekyll/core_ext'
require 'jekyll/site'
require 'jekyll/convertible'
@@ -48,69 +49,6 @@ def require_all(path)
module Jekyll
VERSION = '0.12.0'
- # Default options. Overriden by values in _config.yml or command-line opts.
- # Strings rather than symbols are used for compatability with YAML.
- DEFAULTS = {
- 'safe' => false,
- 'auto' => false,
- 'server' => false,
- 'server_port' => 4000,
-
- 'source' => Dir.pwd,
- 'destination' => File.join(Dir.pwd, '_site'),
- 'plugins' => File.join(Dir.pwd, '_plugins'),
- 'layouts' => '_layouts',
-
- 'future' => true,
- 'lsi' => false,
- 'pygments' => false,
- 'markdown' => 'maruku',
- 'permalink' => 'date',
- 'include' => ['.htaccess'],
- 'paginate_path' => 'page:num',
-
- 'markdown_ext' => 'markdown,mkd,mkdn,md',
- 'textile_ext' => 'textile',
-
- 'maruku' => {
- 'use_tex' => false,
- 'use_divs' => false,
- 'png_engine' => 'blahtex',
- 'png_dir' => 'images/latex',
- 'png_url' => '/images/latex'
- },
-
- 'rdiscount' => {
- 'extensions' => []
- },
-
- 'redcarpet' => {
- 'extensions' => []
- },
-
- 'kramdown' => {
- 'auto_ids' => true,
- 'footnote_nr' => 1,
- 'entity_output' => 'as_char',
- 'toc_levels' => '1..6',
- 'smart_quotes' => 'lsquo,rsquo,ldquo,rdquo',
- 'use_coderay' => false,
-
- 'coderay' => {
- 'coderay_wrap' => 'div',
- 'coderay_line_numbers' => 'inline',
- 'coderay_line_number_start' => 1,
- 'coderay_tab_width' => 4,
- 'coderay_bold_every' => 10,
- 'coderay_css' => 'style'
- }
- },
-
- 'redcloth' => {
- 'hard_breaks' => true
- }
- }
-
# Public: Generate a Jekyll configuration Hash by merging the default
# options with anything in _config.yml, and adding the given options on top.
#
@@ -120,24 +58,7 @@ module Jekyll
#
# Returns the final configuration Hash.
def self.configuration(override)
- # _config.yml may override default source location, but until
- # then, we need to know where to look for _config.yml
- source = override['source'] || Jekyll::DEFAULTS['source']
-
- # Get configuration from <source>/_config.yml
- config_file = File.join(source, '_config.yml')
- begin
- config = YAML.load_file(config_file)
- raise "Invalid configuration - #{config_file}" if !config.is_a?(Hash)
- $stdout.puts "Configuration from #{config_file}"
- rescue => err
- $stderr.puts "WARNING: Could not read configuration. " +
- "Using defaults (and options)."
- $stderr.puts "\t" + err.to_s
- config = {}
- end
-
- # Merge DEFAULTS < _config.yml < override
- Jekyll::DEFAULTS.deep_merge(config).deep_merge(override)
+ config = Configuration.new
+ config.configuration(override)
end
end
View
148 lib/jekyll/configuration.rb
@@ -0,0 +1,148 @@
+module Jekyll
+ # Default options. Overriden by values in _config.yml or command-line opts.
+ # Strings rather than symbols are used for compatability with YAML.
+ DEFAULTS = {
+ 'safe' => false,
+ 'auto' => false,
+ 'server' => false,
+ 'server_port' => 4000,
+
+ 'source' => Dir.pwd,
+ 'destination' => File.join(Dir.pwd, '_site'),
+ 'plugins' => File.join(Dir.pwd, '_plugins'),
+ 'layouts' => '_layouts',
+
+ 'future' => true,
+ 'lsi' => false,
+ 'pygments' => false,
+ 'markdown' => 'maruku',
+ 'permalink' => 'date',
+ 'include' => ['.htaccess'],
+ 'paginate_path' => 'page:num',
+
+ 'markdown_ext' => 'markdown,mkd,mkdn,md',
+ 'textile_ext' => 'textile',
+
+ 'maruku' => {
+ 'use_tex' => false,
+ 'use_divs' => false,
+ 'png_engine' => 'blahtex',
+ 'png_dir' => 'images/latex',
+ 'png_url' => '/images/latex'
+ },
+
+ 'rdiscount' => {
+ 'extensions' => []
+ },
+
+ 'redcarpet' => {
+ 'extensions' => []
+ },
+
+ 'kramdown' => {
+ 'auto_ids' => true,
+ 'footnote_nr' => 1,
+ 'entity_output' => 'as_char',
+ 'toc_levels' => '1..6',
+ 'smart_quotes' => 'lsquo,rsquo,ldquo,rdquo',
+ 'use_coderay' => false,
+
+ 'coderay' => {
+ 'coderay_wrap' => 'div',
+ 'coderay_line_numbers' => 'inline',
+ 'coderay_line_number_start' => 1,
+ 'coderay_tab_width' => 4,
+ 'coderay_bold_every' => 10,
+ 'coderay_css' => 'style'
+ }
+ },
+
+ 'redcloth' => {
+ 'hard_breaks' => true
+ }
+ }
+
+ class Configuration
+
+ def initialize
+ # Keeps the config for further use
+ @merged_config_hash = Jekyll::DEFAULTS
+
+ # Remember all parsed paths to prevent circular inclusion
+ @parsed_paths = []
+ end
+
+ def configuration(override)
+
+ # _config.yml may override default source location, but until
+ # then, we need to know where to look for _config.yml
+ source = override['source'] || Jekyll::DEFAULTS['source']
+
+ # Might not be the best idea, needs discussion.
+ @merged_config_hash['source'] = source
+
+ # Get configuration from <source>/_config.yml
+ config_file = File.join(source, '_config.yml')
+ self.read_and_merge(config_file)
+
+ # finally merge merged_config and override
+ @merged_config_hash.deep_merge(override)
+ end
+
+ # Reads a configuration from a file and merges it into the instance variable.
+ #
+ # @param [String] config_file
+ # @return [Hash]
+ def read_and_merge(config_file)
+
+ begin
+ config = YAML.load_file(config_file)
+ raise "Invalid configuration - #{config_file}" if !config.is_a?(Hash)
+ $stdout.puts "Configuration from #{config_file}"
+
+ if config.has_key? 'additional_configs' and config['additional_configs'].is_a? Array
+ self.process_inclusions(config['additional_configs'], config_file)
+ end
+
+ rescue => err
+ $stderr.puts "WARNING: Could not read configuration. " +
+ "Using defaults (and options)."
+ $stderr.puts "\t" + err.to_s
+ config = {}
+ end
+ @merged_config_hash = @merged_config_hash.deep_merge(config)
+ end
+
+ # Expands a glob to an array of full qualified paths to each config file.
+ #
+ # @param [String] pattern
+ # @return [Array]
+ def glob_to_file_list(pattern, basedir = '')
+ basedir = @merged_config_hash['source'] unless (pattern[0] == '/' or basedir != '')
+ Dir[File.join(basedir, pattern)]
+ end
+
+ # Processes included files.
+ # the parent_file is just as a debug helper for detecting and solving circular references.
+ #
+ # @param [Array] list_of_inclusions
+ # @param [String] parent_file
+ def process_inclusions(list_of_inclusions, parent_file = '')
+ # This is a bit tricky: We want to prevent infinite inclusion loops.
+ # If we add it to the "parsed_path" list right at the top, we do not reflect the true state of the program.
+ # On the other hand, we cannot add it afterwards as the recursion could already happen in the path itself.
+ # Any better ideas than to add it at the start of the processing?
+ list_of_inclusions.each do |path_or_glob|
+ paths = self.glob_to_file_list(path_or_glob)
+ paths.each do |path|
+ if @parsed_paths.include? path
+ $stderr.puts "WARNING: Circular inclusion detected for #{path} in #{parent_file}. Skipping it."
+ next
+ end
+ @parsed_paths << path
+ self.read_and_merge(path)
+ end
+ end
+ end
+ end
+end
View
11 test/fixtures/configuration/one_inclusion/_config.yml
@@ -0,0 +1,11 @@
+permalink: /blog/:year/:month/:day/:title/index.html
+source: source
+destination: public
+plugins: plugins
+code_dir: downloads/code
+category_dir: blog/categories
+markdown: rdiscount
+pygments: false # default python pygments have been replaced by pygments.rb
+
+additional_configs:
+ - additional_configs/_config_a.yml
View
1  test/fixtures/configuration/one_inclusion/additional_configs/_config_a.yml
@@ -0,0 +1 @@
+root: /foobar
View
11 test/fixtures/configuration/pattern_include/_config.yml
@@ -0,0 +1,11 @@
+permalink: /blog/:year/:month/:day/:title/index.html
+source: source
+destination: public
+plugins: plugins
+code_dir: downloads/code
+category_dir: blog/categories
+markdown: rdiscount
+pygments: false # default python pygments have been replaced by pygments.rb
+
+additional_configs:
+ - additional_configs/_config_*.yml
View
1  test/fixtures/configuration/pattern_include/additional_configs/_config_a.yml
@@ -0,0 +1 @@
+root: /foobar
View
1  test/fixtures/configuration/pattern_include/additional_configs/_config_b.yml
@@ -0,0 +1 @@
+root: /buzz
View
12 test/fixtures/configuration/root_config_override/_config.yml
@@ -0,0 +1,12 @@
+root: /overruled
+permalink: /blog/:year/:month/:day/:title/index.html
+source: source
+destination: public
+plugins: plugins
+code_dir: downloads/code
+category_dir: blog/categories
+markdown: rdiscount
+pygments: false # default python pygments have been replaced by pygments.rb
+
+additional_configs:
+ - additional_configs/_config_a.yml
View
1  test/fixtures/configuration/root_config_override/additional_configs/_config_a.yml
@@ -0,0 +1 @@
+root: /foobar
View
27 test/test_configuration.rb
@@ -26,4 +26,31 @@ class TestConfiguration < Test::Unit::TestCase
assert_equal Jekyll::DEFAULTS, Jekyll.configuration({})
end
end
+
+ context "loading configuration with includes" do
+ setup do
+ @replacement_source_dir = File.join(Dir.pwd, 'test', 'fixtures', 'configuration')
+
+ end
+
+ should "include config_a when loading the main config file with one inclusion" do
+ mock($stdout).puts("Configuration from /Users/mario/Dev/Source/jekyll/test/fixtures/configuration/one_inclusion/_config.yml")
+ mock($stdout).puts("Configuration from /Users/mario/Dev/Source/jekyll/test/fixtures/configuration/one_inclusion/additional_configs/_config_a.yml")
+ config = Jekyll.configuration({"source" => File.join(@replacement_source_dir, 'one_inclusion')})
+ assert_equal "/foobar", config["root"]
+ end
+ should "include multiple config files when loading the main config file with one glob inclusion" do
+ mock($stdout).puts("Configuration from /Users/mario/Dev/Source/jekyll/test/fixtures/configuration/pattern_include/_config.yml")
+ mock($stdout).puts("Configuration from /Users/mario/Dev/Source/jekyll/test/fixtures/configuration/pattern_include/additional_configs/_config_a.yml")
+ mock($stdout).puts("Configuration from /Users/mario/Dev/Source/jekyll/test/fixtures/configuration/pattern_include/additional_configs/_config_b.yml")
+ config = Jekyll.configuration({"source" => File.join(@replacement_source_dir, 'pattern_include')})
+ assert_equal "/buzz", config["root"]
+ end
+ should "let the root config overrule all specified configuration files" do
+ mock($stdout).puts("Configuration from /Users/mario/Dev/Source/jekyll/test/fixtures/configuration/root_config_override/_config.yml")
+ mock($stdout).puts("Configuration from /Users/mario/Dev/Source/jekyll/test/fixtures/configuration/root_config_override/additional_configs/_config_a.yml")
+ config = Jekyll.configuration({"source" => File.join(@replacement_source_dir, 'root_config_override')})
+ assert_equal "/overruled", config["root"]
+ end
+ end
end
Something went wrong with that request. Please try again.