Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

hugo mod removes non-Hugo modules from go.mod #6115

Open
deanishe opened this issue Jul 26, 2019 · 19 comments

Comments

@deanishe
Copy link

commented Jul 26, 2019

I use mage to manage my sites (download data, publish via rsync, run pngquant on resized images…), and go mod adds mage's dependencies to go.mod.

Hugo removes them every time I run it.

How do I get Hugo modules to play nice with regular Go modules? Is there an option to make it write its dependencies to, say, hugo.mod instead of the go.mod file?

@bep

This comment has been minimized.

Copy link
Member

commented Jul 26, 2019

Is there an option to make it write its dependencies to, say, hugo.mod instead of the go.mod file?

Not that I know of, but I would be happy to be proven wrong. I would say that I cannot currently fix your problem, but I will think about it.

@bep bep added the Enhancement label Jul 26, 2019

@bep bep modified the milestones: v0.57, v0.56.1 Jul 26, 2019

@bep

This comment has been minimized.

Copy link
Member

commented Jul 26, 2019

OK , thinking a little I think I know what we can do. I have implemented my own "tidy logic" because Go does not understand Hugo's imports. We can adjust that so:

  1. Store away the go.* content
  2. First run go tidy. In most situations this should make them empty.
  3. Apply Hugo's logic, but keep any lefovers from 2)
@deanishe

This comment has been minimized.

Copy link
Author

commented Jul 26, 2019

Sounds like a reasonable workaround. Shame Hugo can't use a different file.

@bep

This comment has been minimized.

Copy link
Member

commented Jul 26, 2019

Shame Hugo can't use a different file.

Or Mage.

image

I have double-checked the Go source, and go.mod is very much hardcoded ... all over the place.

I will take another round on this and check what Go does with these files on regular use, but I suspect that it will be a variant of the above and that with your setup you cannot use "go mod tidy" as that will wipe out the non-Go entries.

@deanishe

This comment has been minimized.

Copy link
Author

commented Jul 26, 2019

Or Mage.

It's not Mage. It's go mod doing its thing.

go.mod is very much hardcoded ... all over the place.

That certainly is very hardcoded. Oh well…

as that will wipe out the non-Go entries

I'm not currently using Hugo modules. But it appears that the presence of a go.mod file causes Hugo to run its tidy routine when building or serving.

Also, wouldn't go mod only remove Hugo's entries if you explicitly run tidy?

@bep bep modified the milestones: v0.56.1, v0.56.2, v0.56.3, v0.57, v0.58 Jul 27, 2019

@deanishe

This comment has been minimized.

Copy link
Author

commented Aug 6, 2019

If this isn't getting added any time soon, could you make Hugo behave the same as go mod and only remove entries from go.mod when explicitly told to do so (or at least only when hugo mod is run)?

Currently, it clears out go.mod every time I build the site or run hugo server, which means I have to undo its changes basically every time I run Hugo (and stop hugo server first, otherwise it'll just remove the entries again).

@bep bep modified the milestones: v0.57, v0.58 Aug 13, 2019

@myitcv

This comment has been minimized.

Copy link

commented Aug 15, 2019

Just run into this as well. I'm not quite sure how Hugo modules are implemented, but I suspect they should be using an approach similar to the way in which tool dependencies are declared?

https://github.com/go-modules-by-example/index/blob/master/010_tools/README.md

i.e. the dependencies need to be declared in Go code if go mod tidy is to see them.

@bep bep modified the milestones: v0.58, v0.57.1 Aug 15, 2019

@bep

This comment has been minimized.

Copy link
Member

commented Aug 15, 2019

i.e. the dependencies need to be declared in Go code if go mod tidy is to see them.

Which is exactly what Hugo does (if you replace "Go code" with "Hugo source"), which is also where the problem lies.

bep added a commit that referenced this issue Aug 15, 2019
@bep

This comment has been minimized.

Copy link
Member

commented Aug 15, 2019

I have "turned off" the tidy during regular builds. WIll be in Hugo 0.57.1. You will still see this problem if you do hugo mod tidy.

@myitcv

This comment has been minimized.

Copy link

commented Aug 15, 2019

which is also where the problem lies.

Just wanted to make sure we're talking about the same thing here: for go mod tidy to work, the dependencies will need to be declared in Go code.

So if you have dependencies declared in "Hugo source" then you will likely need to code generate a build-ignored .go file that also includes those dependencies:

// +build ignore

package hugodeps

import (
	_ "example.com/hugo/dep"
)
@bep

This comment has been minimized.

Copy link
Member

commented Aug 15, 2019

Just wanted to make sure we're talking about the same thing here: for go mod tidy to work, the dependencies will need to be declared in Go code.

We're obviously not. This is not about go mod tidy.

@myitcv

This comment has been minimized.

Copy link

commented Aug 15, 2019

Ah, glad I confirmed. I had understood from the conversation above that the Hugo module dependencies would be added to go.mod, but I guess that is not the case.

@bep

This comment has been minimized.

Copy link
Member

commented Aug 15, 2019

I had understood from the conversation above that the Hugo module dependencies would be added to go.mod, but I guess that is not the case.

They do, but the problem is still not about go mod tidy.

@bep bep modified the milestones: v0.57.1, v0.58 Aug 15, 2019

@deanishe

This comment has been minimized.

Copy link
Author

commented Aug 15, 2019

I had understood from the conversation above that the Hugo module dependencies would be added to go.mod, but I guess that is not the case.

They are. The issue is that Hugo currently behaves as if go.mod is exclusively for its own use and only for Hugo modules, so it removes any actual Go dependencies put there by go mod.

It's also very aggressive about doing that, and "tidies" go.mod when commands other than hugo mod tidy are run. Even when you aren't actually using Hugo modules.

The upshot is that Hugo currently does not play well with go mod.

Not running its tidy routines unless explicitly told to do so would solve the problems this is causing me, but imo, the proper solution is the one @bep initially suggested: Hugo shouldn't alter go mod's entries. After all, the go.mod file belongs to go mod, not Hugo.

@bep

This comment has been minimized.

Copy link
Member

commented Aug 15, 2019

After all, the go.mod file belongs to go mod, not Hugo.

Just to make my point clear:

  • go mod tidy acts as if it owns go.mod, which is probably a bug (considering that Go projects also can include non Go resources, but please don't start a discussion on this here).
  • hugo mod tidy does the same (which is this bug). I have in Hugo 0.57.1 turned off the "auto tidy" feature which should reduce some of this noise.

My main point is this: If you create a Hugo project, that is the spec you need to follow. If you create a "Go and Hugo" mixed module, that comes currently with some issues. If that becomes a problem, put them in each subfolder/module or something.

@myitcv

This comment has been minimized.

Copy link

commented Aug 16, 2019

If you create a Hugo project, that is the spec you need to follow. If you create a "Go and Hugo" mixed module, that comes currently with some issues.

Thanks for clarifying that you distinguish between these two cases. We have a "Go and Hugo" module, hence why I was referring to go mod tidy above.

I think it's worth ensuring that go mod tidy works regardless because this is what people will expect from any project that contains a go.mod. There's more cognitive load in saying "if it's a Hugo project then you need to run hugo mod tidy", especially because that instruction is unintuitive in a "Go and Hugo" mixed module.

go mod tidy could actually be made to work with either type of project. Every time hugo runs it could ensure that a "shadow" code-generated build-ignored .go file reflects the modules referenced in Hugo source. That way, any subsequent run of go mod tidy will see Hugo's requirements reflected via the code-generated .go file.

@deanishe

This comment has been minimized.

Copy link
Author

commented Aug 16, 2019

That way, any subsequent run of go mod tidy will see Hugo's requirements reflected via the code-generated .go file.

That doesn't work because Hugo modules aren't go modules. go mod doesn't understand Hugo modules, and Hugo doesn't understand Go modules.

go mod tidy acts as if it owns go.mod

It's called go.mod, not project.stuff

@myitcv

This comment has been minimized.

Copy link

commented Aug 16, 2019

That doesn't work because Hugo modules aren't go modules.

Then I'm not quite sure I understand why they are being put into go.mod? (#6115 (comment))

@deanishe

This comment has been minimized.

Copy link
Author

commented Aug 16, 2019

Then I'm not quite sure I understand why they are being put into go.mod?

They shouldn't be, imo, but Hugo leverages the go mod machinery to manage its modules, and the go.mod filename is very hardcoded, so Hugo can't easily put its stuff in hugo.mod instead.

In any case, go mod doesn't seem to mind Hugo's modules being in there, although go mod tidy will remove them. Now that Hugo v0.57.1 isn't removing go mod's stuff all the time, it's no longer an annoying problem in practice.

Thanks @bep!

@bep bep modified the milestones: v0.58, v0.59, v0.60 Sep 2, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
3 participants
You can’t perform that action at this time.