Collection relative paths

[mathematicalcoffee]

Addressing this issue. #2518

Aim: allow the directory that a collection is in to be different to <source>/_<collection name>.

Reason: e.g. for cleanliness of the top-level directory - we could have _collections/ and list all of our collections under there.

Achieved by: allow the user to set the relative_directory property of a collection in _config.yml with the path to the collection's directory.

For example:

Example:

Suppose I had a collection my_collection. Before this patch, it would have to be in <source_directory>/_my_collection. Suppose now I move it to _collections/stupid_name.

In _config.yml, just add relative_directory to my_collection's metadata:

collections:
  my_collection:
    relative_directory: _collections/stupid_name

The resulting collection:

• has label my_collection (regardless of the name of the actual directory name, stupid_name in this case - though in practice you'd keep the name of the directory the same as the name of the collection).
• is copied to <dest>/<collection name>/
• each document in the collection has path property _collections/stupid_name/index.md, rather than _my_collection/index.md. This is the only thing I might think of as being inconsistent.

Questions

Are the tests rigorous enough? And what about behaviour re: the last point?

Note - in the 'master' version of jekyll this path property would be the absolute path to the (source) file, e.g. /home/amy/sandbox/jekyllsite/_my_collection/index.md. In this PR it changes to the relative path _collections/stupid_name/index.md.

The jekyll documentation says path should be

Path to the document relative to the collection's directory.

According to this, the path property should really be index.md (being relative to the collection directory), so both the current jekyll and this PR jekyll are getting it wrong - the former gives an absolute path, and the later gives a relative path, but to the site's directory (not the collection's directory).

But perhaps that should be a separate issue (whereby either the docs are updated or the meaning of path is made clear), particularly as changing path would probably break many people's sites.

[fniephaus]

+1 this is great!

[envygeeks]

Is there anything holding this back @jekyll/core because this is a good idea? I went looking explicitly for this just now because I'm having the same organization aches with collections now that I have a site with dozens upon dozens of collections.

[mattr-]

IMO, this would have to be something where you can only change the name of the directory that Jekyll should look for the collection within the source path (i.e. collections must be sandboxed within the main source directory). I haven't looked at the code yet to see if that's what this does, but it does sound like a good idea.

[envygeeks]

That's is definitely something not done in the source thus far, I'm going to ping @mathematicalcoffee and ask that a test + alteration be made to the variable that it accepts and use Jekyll.sanitized_path on the result before allowing it to be put into a variable that's cached. This should suffice to suite what @mattr- wants.

[envygeeks]

I really shouldn't say what @mattr- wants more than what Jekyll needs because of Github Pages.

[mattr-]

@envygeeks "Because GitHub Pages" was indeed the reason behind requiring the containment within the site source directory. 🤘

[parkr]

"Because GitHub Pages" was indeed the reason behind requiring
the containment within the site source directory. 🤘

😄 Indeed, they'll require this patch in order to use this code.

[mathematicalcoffee]

Hi folks - I think I've enforced this (using site.in_source_directory which calls Jekylll.sanitized_path), but I am really not sure about how to test it, or if I've enforced it properly.
Sorry, in the couple of weeks since originally submitting the PR my "jekyll-brain" has gone to mush, and I don't feel at all familiar with the code any more.

Let me know if what has been done is OK, or if you could help me out with the additional test (or you are welcome to do it too).

[envygeeks]

I'll look into adding tests this week sometime if I have time, I don't know if I'll have time so this might have to wait a few weeks but I'll definitely make sure it gets done before we are close to Jekyll3. Thanks for making the changes.

[kristianb21]

+1 sweet deal!

[parkr]

Your code uses relative_directory, not relative_path.

[parkr]

Still interested in seeing this?

I was thinking more along the lines of "here's where all my collections are". Instead of collections just being in ./_collection_name, it'd be in ./relative_directory/_collection_name. And they'd all be there, nested together.

[envygeeks]

I'm still 👍 on this, I have a site that has like 30 collections and it would be great to be able to nest them in a dedicated folder to try and cleanup our root dir, but it's not necessarily a big deal either for me because we backported this into the plugins directory since it's not built by Github.

[mathematicalcoffee]

I'd still like this. I personally prefer the "here's where this specific collection is" approach, but I do see merit to the "here's where all my collections are" approach, for sure, and I'll take whatever I can get.

My use case to prefer the former is:

• all my plants have a collection each, in _plants/{plant id}. (to avoid cluttering the root dir with all the plants I have)
• I have a separate collection _cultural-notes with genus cultural notes, and I might have a separate collection _fertilisers with my notes on various fertilisers, etc..

That being said the second approach would be fine, so long as I could nest my collections in subdirectories as I wished, e.g relative_directory/_plants/{plant id} are each individual collections as are relative_directory/_cultural-notes and relative_directory/_fertilisers.

[sen-lu]

+1 this would be fantastic to have

[mathematicalcoffee]

In the meantime, you can just have this behaviour as a jekyll plugin. Just make a file (e.g.) subcollections.rb in _plugins:

module Jekyll
  class Collection
    def relative_directory
      @relative_directory ||= (metadata['relative_directory'] && site.in_source_dir(metadata['relative_directory']) ||  "_#{label}")
    end
  end

  # you only need to put this in if you are using jekyll < 2.5.0 (e.g. github-pages currently using 2.4)
  # it's just copied from the latest jekyll.
  class Site
    if !Jekyll::Site.method_defined? :in_source_dir
      # Public: Prefix a given path with the source directory.
      #
      # paths - (optional) path elements to a file or directory within the
      #         source directory
      #
      # Returns a path which is prefixed with the source directory.
      def in_source_dir(*paths)
        paths.reduce(source) do |base, path|
          Jekyll.sanitized_path(base, path)
        end
      end
    end
  end
end

I have found I almost always want to set permalink on my subcollections (in _config.yml) because of the path confusion mentioned in the first post.

[parkr]

@mathematicalcoffee For Jekyll 3, let's just do the "here's where all my collections are." If it's insufficient, we can update it in the future.

[parkr]

Want me to take a stab at the simpler version?

[k-funk]

@mathematicalcoffee is this intended to work with output: true? You mentioned setting a permalink, but the :path includes the directory (_pages in my example):

collections:
  my_collection:
    relative_directory: '_pages/stupid_name'
    output: true

But that produces:

_pages/
└── stupid_name/
    └── some_page.html

_site/
└── my_collection/
    └── _pages/
        └── stupid_name/
            └── some_page.html

Which is the same as permalink: /:path
I want

_site/
└── my_collection/
    └── some_page.html

[maxg203]

This feature would be really nice to have and a good implementation as far as I can tell. Is there any reason that this has not been merged yet?

[mathematicalcoffee]

We have not received nor merged a PR which allows you to specify a directory either for all collections, or on a per-collection basis. We are still interested in the former, and the latter is seeming more appealing as time moves onward.

(parkr's comment 1/Mar/2016).

I was under the impression that this PR achieves the latter. I got a notification that this pull request was closed - does that mean that jekylll now has either of these features, almost 2 years after the pull request was submitted?

[maxg203]

@mathematicalcoffee I agree, it seems that although this PR seems to allow you to specify a directory, this issue has been closed but the PR not merged. Can I ask a member like @envygeeks or @parkr why? I'd actually argue this should be open until it is merged.

[ranuss]

Why isn't this feature merged yet? 😐

[maxg203]

Can someone merge this please?

[parkr]

I'll reevaluate this PR soon.

[parkr]

Ok I definitely still want a feature like this. I think being able to nest collections would be nice. I remember talking with @budparr about having a site-wide subfolder, like collections_dir. By default it's just the root of the site source, but you could set it to _my_collections and we'd look in _my_collections/_posts for posts, etc. I think configuring each collection individually puts a huge burden on the user. I'd like to see if the intermediate solution of just having a collections_dir would work for most people.

[maxg203]

@parkr Yes, even that would definitely be a nice improvement. Thanks for taking a look.

[parkr]

Cool. A top-level configuration option, collections_dir, would work here. @mathematicalcoffee, would you be open to reworking this PR to use a centralized collections dir instead of a separate dir configuration for each collection?

[parkr]

I have a replacement for this in #6331 which allows the user to set 1 subdirectory for all collections.

[DirtyF]

closing in favor of #6331

@mathematicalcoffee thanks for the insight ❤️

[hosnas]

Can we have both of these features at the same time please? I mean we could be able to define a subdirectory for all collections, and also we would be able to overwrite it with relative_directory for any specific collection that we want to put somewhere else. @parkr @DirtyF

[DirtyF]

@hosnas Did you try with the aforementioned plugin?

[adavidramos]

I think that the plugin is now broken in v3.7.0. Can't troubleshoot at the moment, but will update if I learn more.

[parkr]

@adavidramos This is now built-into Jekyll in 3.7.0. Just set collections_dir in your _config.yml file.

[hosnas]

@DirtyF as @adavidramos suggests, it seems the plugin is broken. Can we have an option to overwrite collections_dir for some specific collections as a feature in jekyll?