Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

proposal: cmd/go: support local directive in go.mod #51779

Closed
liubog2008 opened this issue Mar 18, 2022 · 10 comments
Closed

proposal: cmd/go: support local directive in go.mod #51779

liubog2008 opened this issue Mar 18, 2022 · 10 comments

Comments

@liubog2008
Copy link

liubog2008 commented Mar 18, 2022

Why

One go.mod is not reasonable for monorepo

For example, we have a repo with structure as below

- cmd
- pkg
  - server
  - client
  - api
- go.mod
- go.sum

If the repo consumers hope to import client and api pkg, they have to import the whole module and indirectly import dependencies of server pkg.

Only three choices for this situation.

  1. New git repo for client and api
  2. New go.mod file and new git tag for client and api, e.g client/v0.1.0
  3. New go.mod file and use replace directive

For choice 1, we have to manage more git repos and submit more PRs for a feature.
For choice 2, we have to tag api and client, then change the go.mod version and tag main module.
For choice 3, replace directive has some problems now.

replace directive for monorepo

The below lines are used in many go.mod files.

replace github.com/xxx/yyy/api => ./api
replace github.com/xxx/yyy/client => ./client

However, go install is not worked if replace directive exists(See #44840, #40276)

Another problem of replace is if client also imports api and replaces it. consumers have to use replace in go.mod again to ensure api module's version.

require (
    github.com/xxx/yyy/api v0.0.1
    github.com/xxx/yyy/client v0.0.1
)

# If omit this line, the below error will be thrown
# go: github.com/xxx/yyy/client@v0.0.1 requires
#.     github.com/xxx/yyy/api@v0.0.0-00010101000000-000000000000: invalid version: unknown revision 000000000000
replace github.com/xxx/yyy/api => github.com/xxx/yyy/api v0.0.1

Design

  1. Add local directive for all modules which are imported from same git repo.
  2. Use same version as the main module to import local modules.

In main module

module github.com/xxx/yyy

local (
    github.com/xxx/yyy/client
    github.com/xxx/yyy/api
)

in client module

module github.com/xxx/yyy/client

local (
    github.com/xxx/yyy/api
)

in consumer module

module github.com/aaa/bbb

require (
    github.com/xxx/yyy/client v0.0.1
    github.com/xxx/yyy/api v0.0.1
)

go mod graph for consumer

github.com/aaa/bbb github.com/xxx/yyy/client@v0.0.1
github.com/aaa/bbb github.com/xxx/yyy/api@v0.0.1
github.com/xxx/yyy/client@v0.0.1 github.com/xxx/yyy/api@v0.0.1

Compatibility

No compatibility problem

Unresolved Issues

Git tag client/v0.0.1 will be conflict with v0.0.1in semantic. There are two solutions:

  1. client/v0.0.1 will override the v0.0.1 tag.
  2. Throw error if they are both used. (I prefer)

Alternatives

  1. Just omit version in require as the local module. (also good to me)
  2. Change the behavior of replace directive.
@gopherbot gopherbot added this to the Proposal milestone Mar 18, 2022
@ianlancetaylor ianlancetaylor added this to Incoming in Proposals (old) Mar 18, 2022
@ianlancetaylor
Copy link
Contributor

CC @bcmills

@seankhliao seankhliao changed the title proposal: x/mod: support local directive in go.mod proposal: cmd/go: support local directive in go.mod Mar 18, 2022
@beoran
Copy link

beoran commented Mar 18, 2022

Go 1.18 has workspaces. Could they be used in your use case?

@mvdan
Copy link
Member

mvdan commented Mar 18, 2022

Also note that this is otherwise a duplicate of #26640.

@liubog2008
Copy link
Author

Go 1.18 has workspaces. Could they be used in your use case?

go.work seems a local workspace and not be pushed to remote repo in best practice? If I split monorepo to three git repos, go.work can be easily used for me to modify my code.

@liubog2008
Copy link
Author

Also note that this is otherwise a duplicate of #26640.

I feel they are not duplicated.

@bandesz
Copy link

bandesz commented Aug 3, 2022

I would like to add a further issue to consider here.

We have a similar project structure, but we also use vendoring in the root project. Using the example from the description, this means the pkg/client module is copied to vendor (but with full module path of course), and whenever you make a change to pkg/client during development, you have to run go mod vendor.

@liubog2008
Copy link
Author

liubog2008 commented Apr 24, 2023

Any feedback about this proposal?
I think it's useful for many go projects to removereplace in their go.mod.
For example, the popular KV store ETCD.
We can remove replace in https://github.com/etcd-io/etcd/blob/main/go.mod#L5-L15 and use go install to install etcdctl

@rsc
Copy link
Contributor

rsc commented May 3, 2023

This proposal has been added to the active column of the proposals project
and will now be reviewed at the weekly proposal review meetings.
— rsc for the proposal review group

@rsc
Copy link
Contributor

rsc commented May 10, 2023

As envisioned above, this feature would be incompatible with modules. The specific goal seems to be to have something that is like replace but that applies during go install. But applying during go install means applying during any kind of version selection decisions, and it would impose a constraint of the form "use exactly this version" instead of "use this version or newer". Allowing that new kind of constraint changes version selection from trivial to NP-complete, which we're not going to do.

A separate problem would be how to infer the actual version to use. The go command does not fundamentally operate on Git repositories, and there is no way to express "use the same Git commit for m/sub as for m". So even if that constraint were not computationally problematic, it still depends on breaking down abstractions that we currently have.

To work on a collection of related modules in a single repository and have the convenience of having them all bound together while working in a git clone of the repo, the answer is go.work.

The answer for making go install work is to tag the sub-modules, then update the appropriate require lines in the command module, then tag the command module.

@rsc
Copy link
Contributor

rsc commented May 10, 2023

This proposal has been declined as infeasible.
— rsc for the proposal review group

@rsc rsc closed this as completed May 10, 2023
@golang golang locked and limited conversation to collaborators May 9, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
No open projects
Development

No branches or pull requests

8 participants