Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add a way to remap/mount Hugo Components #6010

Closed
bep opened this issue Jun 1, 2019 · 21 comments

Comments

@bep
Copy link
Member

commented Jun 1, 2019

Related to #5996 etc.
A Hugo Component has a fixed set of folders where the folder tells Hugo how to deal with it (layouts, static, assets)...

With the upcoming Hugo Modules ™ it becomes very easy to import components stored in GitHub repos etc. by simply importing them.

We require very little for a repo to become importable:

  • It would be great if it had a semver tag, but that's not a must
  • If the component itself wants to import others (transitive) it must have a config.toml.
  • It will need to have one of the folders in question to do anything useful.

So, if I wanted to import Bootstrap as a SCSS module, I would want https://github.com/twbs/bootstrap/tree/master/scss mounted below /assets.

There is currently no way to express that, but it would be really useful if we could.

Something ala:

[module]
[[imports]]
path="github.com/twbs/bootstrap"
mounts=[[ "scss", "assets/bootstrap/scss" ]]

I can add to this that there will be a default mount list if not set above (static, assets ...), and for vendoring purposes we will stick to that list, which is a great way to just filter what you need from a dependency..

/cc @regisphilibert @kaushalmodi @moorereason @onedrawingperday @digitalcraftsman and gang.

@bep bep added this to the v0.56 milestone Jun 1, 2019

@bep bep self-assigned this Jun 1, 2019

@bep bep added the Enhancement label Jun 1, 2019

@kaushalmodi

This comment has been minimized.

Copy link
Member

commented Jun 1, 2019

It would be single square brackets in mounts=[[ "scss", "assets/bootstrap/scss" ]] .

But what does the first "scss" value imply. I understand that the second argument is probably where you want to clone the repo.

@kaushalmodi

This comment has been minimized.

Copy link
Member

commented Jun 1, 2019

May be this:

[module] 
  [[imports]] 
    source_url = "github.com/twbs/bootstrap"
    source_dir = "scss"
    mount_point = "assets/bootstrap/scss"
@bep

This comment has been minimized.

Copy link
Member Author

commented Jun 1, 2019

@kaushalmodi it's an array of tuples, so the syntax is correct. And this is not about "where to clone". There can be potentially many such mount points for a given import and this is just a way to set up Hugo's big virtual filesystem. In my example, you would be able to do:

{{ resources.Get "bootstrap/scss/bootstrap.scss" | toCSS }}
@larzza

This comment has been minimized.

Copy link

commented Jun 1, 2019

There is currently no way to express that, but it would be really useful if we could.

@bep Do you mean ”no way to express that in” Hugo Modules ™ ;-) or in Hugos virtual FS or are you thinking of something else?

Read the description once more and I guess you just mean what this issue is all about.

@bep

This comment has been minimized.

Copy link
Member Author

commented Jun 1, 2019

I'm not sure about the question, but the above hasn't been specified/implemented and ergo not possible. But it's possible to implement, and it will be superuseful.

So, what you get by default for a componet having a layouts and static folder is this:

[module]
[[imports]]
path="github.com/bep/mypartials"
mounts=[[ "layouts", "layouts" ], [ "static", "static" ]]

The above is implicit and you don't have to define it. The value to the right must start with the predefined set of folders (assets, static ...)

@larzza

This comment has been minimized.

Copy link

commented Jun 1, 2019

Yes, it will indeed be super usefull!

But do you really mean this?

It will need to have one of the folders in question to do anything useful.

I would expect to be able to import what ever I want and mount it to one of the predefined destinations ”assets” etc...

Given a repository with the directory cake at the top level, maybe something like this?

Import:
  - github.com/bake/hugo/v4 v4.3.1 as hugo
  - another import as another
Mount:
  - from: hugo, source: cake
  - target: assets/scss/mysassycss
@bep

This comment has been minimized.

Copy link
Member Author

commented Jun 1, 2019

It will need to have one of the folders in question to do anything useful.

Yes, I meant that. Which is how you would create your Hugo Component from scratch, using that naming standard. This particular issue is about being able to use non-Hugo-components (aka Bootstrap).

I will leave this discussion to others now; unless you find something obviously wrong with my syntax, I don't think we're going to take the whole module/imports/package discussion all over again.

@larzza

This comment has been minimized.

Copy link

commented Jun 1, 2019

Hm, I wonder if you misunderstand me (and I you)? My example was only meant to explain what I was aiming at, not to suggest a different syntax. I understand that this is about Bootstrap and the like (which is something I really want as I think you remeber?). Bootstrap happens to have a subdirectory named scss, but that might not always be the case, thats why I suggested to be able to mount an arbitrary source to one of the predefined destinations handled by Hugo.

To ”to take the whole module/imports/package discussion all over again.” is the last thing that I want.

Yes, let’s leave the disussion to others.

@regisphilibert

This comment has been minimized.

Copy link

commented Jun 2, 2019

I don't have much to say. Looks lovely. And wouldn't it remove the use for #5973 now that we could

  • imports non Hugo components
  • map directories

Ex:

module:
  imports:
   - path: github.com/my-english-content
     mount: src/markdown > content/en
   - path: github.com/my-french-content
     mounts: src/markdown > content/fr

Pardon the yaml above, I don't know what tuples are...

@bep

This comment has been minimized.

Copy link
Member Author

commented Jun 2, 2019

@regisphilibert I suspect content will be on a blacklist in its first iteration (there are more to that particular thing than just the mapping, so it needs more work), but in "theory" you are right.

I don't think YAML supports arrays with more than one dimension, so we might as well do it like this:

[module]
[[imports]]
path="github.com/bep/mycomponent"
[[imports.mounts]]
source="scss"
target="assets/bootstrap/scss"
[[imports.mounts]]
source="src/markdown"
target="content/en"
language="en"

With the above, we set aside room for additional metadata ....

@larzza

This comment has been minimized.

Copy link

commented Jun 2, 2019

@bep This is what I tried to discuss yesterday when you seemed to disagree.

[[imports.mounts]]
source="src/markdown" <=== no need to name dir by one of the ”Hugo dirs”.
target="content/en" <=== need to be one of the ”Hugo dirs” (content not initially supported)
language="en"
@bep

This comment has been minimized.

Copy link
Member Author

commented Jun 2, 2019

@larzza OK, I guess that pseudo syntax confused me ... Or maybe it was the cake :-)

@bep

This comment has been minimized.

Copy link
Member Author

commented Jun 3, 2019

A note:

I'm going to try to get content as part of this in the upcoming release. The gain is just too big to not do it, it's going to be super useful, esp. for those bigger multilingual sites. But it's going to look something like this:

[[module.imports.mounts]]
source="src/markdown/blog"
target="content/blog"
language="en"

So, no language indicator in the paths (source, target), which means you can do pretty flexible remapping of any of your component type (content, static, layouts ...). The above example would put the content in the blog section.

bep added a commit that referenced this issue Jun 3, 2019

Add Hugo Modules
Fixes #5973
Fixes #5996
Fixes #6010
Fixes #5911
Fixes #5940
@regisphilibert

This comment has been minimized.

Copy link

commented Jun 3, 2019

So if we add language: en, Hugo will look for the content files flagged as en (weather by directory or filename) and only "add" those to the module?

I presume without the key, all will be added?

@regisphilibert

This comment has been minimized.

Copy link

commented Jun 3, 2019

In all of this, I didn't get how you could point to particular release/tag of the given repo, when importing.

@bep

This comment has been minimized.

Copy link
Member Author

commented Jun 3, 2019

So if we add language: en,

No, all will be added, the language is just to tell Hugo what language they're in (if not in the filename, e.g. foo.en.md.

In all of this, I didn't get how you could point to particular release/tag of the given repo, when importing.

Hugo just wraps
https://github.com/golang/go/wiki/Modules

So most things in that document applies, including https://github.com/golang/go/wiki/Modules#version-selection

When you import a module in config.toml you will get the version selected by the minimal version selection algorithm (usually the latest main version).

From there there are several options:

Update All

hugo mod get -u

Update one

hugo mod get -u github.com/bep/hugotestmods/mypartials
hugo mod get -u github.com/bep/hugotestmods/mypartials@v1.0.1
hugo mod get -u github.com/bep/hugotestmods/mypartials@master

Etc.

And from the example above it's a multi-module repo, updating only one of them.

@regisphilibert

This comment has been minimized.

Copy link

commented Jun 3, 2019

Sorry if my questions come bit late, I only recently found time to digest this.

It seems the .mod file in a usual Go Module project gives version information on any given "import". So in a Hugo Project context, will we have some sort of go.mod equivalent so that the versions of the different modules can be committed somewhere?

@regisphilibert

This comment has been minimized.

Copy link

commented Jun 3, 2019

No, all will be added, the language is just to tell Hugo what language they're in (if not in the filename, e.g. foo.en.md.

Wouldn't that fall into the responsability of the person building the module.

If the remote content repo holds every languages seperated by directories, or by filename:

[[module.imports.mounts]]
source="src/markdown/blog"
target="content/blog"

If the remote content repo holds only the english files without any lang info in the filename:

[[module.imports.mounts]]
source="src/markdown/blog"
target="content/en/blog/"

I understand we want to safegard users, but using a remote repo for content management means you're aware of how Hugo Mutlilingual work, or am I missing a use case here?

@larzza

This comment has been minimized.

Copy link

commented Jun 4, 2019

It seems the .mod file in a usual Go Module project gives version information on any given "import". So in a Hugo Project context, will we have some sort of go.mod equivalent so that the versions of the different modules can be committed somewhere?

Hugo Modules use Go modules under the hood so go.mod and go.sum will be available.

  • It would be nice to be able to add version information in site config, but I don’t think this will be possible?
  • Will Hugo Modules support import of different major versions?

@bep

@bep

This comment has been minimized.

Copy link
Member Author

commented Jun 4, 2019

@regisphilibert the content part here works (will work) exactly as content works today. So you can still do myarticle.en.md etc. But the big new use case in all of this is that you can split translations into modules and let the Dutch people, Japanese people, Norwegian people each work on their own repository (with their own permissions) -- and then they are done they can create a PR to the main repo: "Here are all the translations!". For these people we need a way to tell what language the module is (mostly) in (exactly the same as today when you put contentDir below a given language).

Go Modules is too big to discuss here. I can just say that I'm mostly very, very impressed by it. It is the dependency/management up until now that has impressed me the most.

It would be nice to be able to add version information in site config, but I don’t think this will be possible?

No, at least not now, and I don't see it either.

Will Hugo Modules support import of different major versions?

You can import v2 versions etc., but you can currently not have v1 and v2 of a given module in play at once. I have thought long and hard about it, and it definitively need much more long and hard thoughts (we could add some namespace support, but we soon get into to troubles in the layouts section) ... So that will have to wait.

@larzza

This comment has been minimized.

Copy link

commented Jun 4, 2019

You can import v2 versions etc., but you can currently not have v1 and v2 of a given module in play at once. I have thought long and hard about it, and it definitively need much more long and hard thoughts (we could add some namespace support, but we soon get into to troubles in the layouts section) ... So that will have to wait.

I guess it might not makes much sense to have support for different versions at once in the Hugo context anyway, or at least makes things a bit to complicated for the theme developer.

bep added a commit that referenced this issue Jun 10, 2019

Add Hugo Modules
Fixes #5973
Fixes #5996
Fixes #6010
Fixes #5911
Fixes #5940

bep added a commit that referenced this issue Jun 10, 2019

Add Hugo Modules
Fixes #5973
Fixes #5996
Fixes #6010
Fixes #5911
Fixes #5940

bep added a commit that referenced this issue Jun 10, 2019

Add Hugo Modules
Fixes #5973
Fixes #5996
Fixes #6010
Fixes #5911
Fixes #5940

bep added a commit that referenced this issue Jun 11, 2019

Add Hugo Modules
Fixes #5973
Fixes #5996
Fixes #6010
Fixes #5911
Fixes #5940

bep added a commit that referenced this issue Jun 16, 2019

Add Hugo Modules
Fixes #5973
Fixes #5996
Fixes #6010
Fixes #5911
Fixes #5940

bep added a commit that referenced this issue Jun 18, 2019

Add Hugo Modules
Fixes #5973
Fixes #5996
Fixes #6010
Fixes #5911
Fixes #5940

bep added a commit that referenced this issue Jun 18, 2019

Add Hugo Modules
Fixes #5973
Fixes #5996
Fixes #6010
Fixes #5911
Fixes #5940

bep added a commit that referenced this issue Jun 20, 2019

Add Hugo Modules
Fixes #5973
Fixes #5996
Fixes #6010
Fixes #5911
Fixes #5940

bep added a commit that referenced this issue Jun 21, 2019

Add Hugo Modules
Fixes #5973
Fixes #5996
Fixes #6010
Fixes #5911
Fixes #5940

bep added a commit that referenced this issue Jun 26, 2019

Add Hugo Modules
Fixes #5973
Fixes #5996
Fixes #6010
Fixes #5911
Fixes #5940

bep added a commit that referenced this issue Jun 26, 2019

Add Hugo Modules
Fixes #5973
Fixes #5996
Fixes #6010
Fixes #5911
Fixes #5940

bep added a commit that referenced this issue Jun 28, 2019

Add Hugo Modules
Fixes #5973
Fixes #5996
Fixes #6010
Fixes #5911
Fixes #5940

bep added a commit that referenced this issue Jul 3, 2019

Add Hugo Modules
Fixes #5973
Fixes #5996
Fixes #6010
Fixes #5911
Fixes #5940

bep added a commit that referenced this issue Jul 5, 2019

Add Hugo Modules
Fixes #5973
Fixes #5996
Fixes #6010
Fixes #5911
Fixes #5940

bep added a commit that referenced this issue Jul 6, 2019

Add Hugo Modules
Fixes #5973
Fixes #5996
Fixes #6010
Fixes #5911
Fixes #5940
Fixes #6074

bep added a commit that referenced this issue Jul 10, 2019

Add Hugo Modules
Fixes #5973
Fixes #5996
Fixes #6010
Fixes #5911
Fixes #5940
Fixes #6074
Fixes #6082

bep added a commit that referenced this issue Jul 10, 2019

Add Hugo Modules
Fixes #5973
Fixes #5996
Fixes #6010
Fixes #5911
Fixes #5940
Fixes #6074
Fixes #6082

bep added a commit that referenced this issue Jul 17, 2019

Add Hugo Modules
Fixes #5973
Fixes #5996
Fixes #6010
Fixes #5911
Fixes #5940
Fixes #6074
Fixes #6082

bep added a commit that referenced this issue Jul 19, 2019

Add Hugo Modules
Fixes #5973
Fixes #5996
Fixes #6010
Fixes #5911
Fixes #5940
Fixes #6074
Fixes #6082
Fixes #6092

bep added a commit that referenced this issue Jul 21, 2019

Add Hugo Modules
Fixes #5973
Fixes #5996
Fixes #6010
Fixes #5911
Fixes #5940
Fixes #6074
Fixes #6082
Fixes #6092

bep added a commit that referenced this issue Jul 22, 2019

Add Hugo Modules
Fixes #5973
Fixes #5996
Fixes #6010
Fixes #5911
Fixes #5940
Fixes #6074
Fixes #6082
Fixes #6092

bep added a commit that referenced this issue Jul 23, 2019

Add Hugo Modules
Fixes #5973
Fixes #5996
Fixes #6010
Fixes #5911
Fixes #5940
Fixes #6074
Fixes #6082
Fixes #6092

bep added a commit that referenced this issue Jul 24, 2019

Add Hugo Modules
This commit implements Hugo Modules.

This is a broad subject, but some keywords include:

* A new `module` configuration section where you can import almost anything. You can configure both your own file mounts nd the file mounts of the modules you import. This is the new recommended way of configuring what you earlier put in `configDir`, `staticDir` etc. And it also allows you to mount folders in non-Hugo-projects, e.g. the `SCSS` folder in the Bootstrap GitHub project.
* A module consists of a set of mounts to the standard 7 component types in Hugo: `static`, `content`, `layouts`, `data`, `assets`, `i18n`, and `archetypes`. Yes, Theme Components can now include content, which should be very useful, especially in bigger multilingual projects.
* Modules not in your local file cache will be downloaded automatically and even "hot replaced" while the server is running.
* Hugo Modules supports and encourages semver versioned modules, and uses the minimal version selection algorithm to resolve versions.
* A new set of CLI commands are provided to manage all of this: `hugo mod init`,  `hugo mod get`,  `hugo mod graph`,  `hugo mod tidy`, and  `hugo mod vendor`.

All of the above is backed by Go Modules.

Fixes #5973
Fixes #5996
Fixes #6010
Fixes #5911
Fixes #5940
Fixes #6074
Fixes #6082
Fixes #6092

bep added a commit that referenced this issue Jul 24, 2019

Add Hugo Modules
This commit implements Hugo Modules.

This is a broad subject, but some keywords include:

* A new `module` configuration section where you can import almost anything. You can configure both your own file mounts nd the file mounts of the modules you import. This is the new recommended way of configuring what you earlier put in `configDir`, `staticDir` etc. And it also allows you to mount folders in non-Hugo-projects, e.g. the `SCSS` folder in the Bootstrap GitHub project.
* A module consists of a set of mounts to the standard 7 component types in Hugo: `static`, `content`, `layouts`, `data`, `assets`, `i18n`, and `archetypes`. Yes, Theme Components can now include content, which should be very useful, especially in bigger multilingual projects.
* Modules not in your local file cache will be downloaded automatically and even "hot replaced" while the server is running.
* Hugo Modules supports and encourages semver versioned modules, and uses the minimal version selection algorithm to resolve versions.
* A new set of CLI commands are provided to manage all of this: `hugo mod init`,  `hugo mod get`,  `hugo mod graph`,  `hugo mod tidy`, and  `hugo mod vendor`.

All of the above is backed by Go Modules.

Fixes #5973
Fixes #5996
Fixes #6010
Fixes #5911
Fixes #5940
Fixes #6074
Fixes #6082
Fixes #6092

bep added a commit that referenced this issue Jul 24, 2019

Add Hugo Modules
This commit implements Hugo Modules.

This is a broad subject, but some keywords include:

* A new `module` configuration section where you can import almost anything. You can configure both your own file mounts nd the file mounts of the modules you import. This is the new recommended way of configuring what you earlier put in `configDir`, `staticDir` etc. And it also allows you to mount folders in non-Hugo-projects, e.g. the `SCSS` folder in the Bootstrap GitHub project.
* A module consists of a set of mounts to the standard 7 component types in Hugo: `static`, `content`, `layouts`, `data`, `assets`, `i18n`, and `archetypes`. Yes, Theme Components can now include content, which should be very useful, especially in bigger multilingual projects.
* Modules not in your local file cache will be downloaded automatically and even "hot replaced" while the server is running.
* Hugo Modules supports and encourages semver versioned modules, and uses the minimal version selection algorithm to resolve versions.
* A new set of CLI commands are provided to manage all of this: `hugo mod init`,  `hugo mod get`,  `hugo mod graph`,  `hugo mod tidy`, and  `hugo mod vendor`.

All of the above is backed by Go Modules.

Fixes #5973
Fixes #5996
Fixes #6010
Fixes #5911
Fixes #5940
Fixes #6074
Fixes #6082
Fixes #6092

bep added a commit that referenced this issue Jul 24, 2019

Add Hugo Modules
This commit implements Hugo Modules.

This is a broad subject, but some keywords include:

* A new `module` configuration section where you can import almost anything. You can configure both your own file mounts nd the file mounts of the modules you import. This is the new recommended way of configuring what you earlier put in `configDir`, `staticDir` etc. And it also allows you to mount folders in non-Hugo-projects, e.g. the `SCSS` folder in the Bootstrap GitHub project.
* A module consists of a set of mounts to the standard 7 component types in Hugo: `static`, `content`, `layouts`, `data`, `assets`, `i18n`, and `archetypes`. Yes, Theme Components can now include content, which should be very useful, especially in bigger multilingual projects.
* Modules not in your local file cache will be downloaded automatically and even "hot replaced" while the server is running.
* Hugo Modules supports and encourages semver versioned modules, and uses the minimal version selection algorithm to resolve versions.
* A new set of CLI commands are provided to manage all of this: `hugo mod init`,  `hugo mod get`,  `hugo mod graph`,  `hugo mod tidy`, and  `hugo mod vendor`.

All of the above is backed by Go Modules.

Fixes #5973
Fixes #5996
Fixes #6010
Fixes #5911
Fixes #5940
Fixes #6074
Fixes #6082
Fixes #6092

@bep bep closed this in #5960 Jul 24, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
4 participants
You can’t perform that action at this time.