Go modules support #15868
Assignees
Labels
Comments
Merged
andresrc
added
[zube]: Inbox
Team:Services
|
The following packages lack version information:
All of the packages contain a LICENSE file which is included in our |
This was referenced Feb 20, 2020
kvch
added a commit
that referenced
this issue
Mar 3, 2020
This PR introduces the new dependency management system for beats. From now on we are using `go mod` instead of `govendor`. The name of the module we provide is named `github.com/elastic/beats/v7`.
We adopted go modules, but we still keep dependencies under the folder `vendor`. However, it is maintained by `go mod` now by running `mage vendor`. Thus, it is not possible to add local changes to the dependencies there because next time someone runs the command it will be overwritten. **If you need to apply patches to dependencies either fork the repository or try to contribute it back to the original repo.**
This PR does not address the changes in Beat generators. Those are going to be updated in a follow-up PR.
### FAQ
#### How can I add a new dependency?
TL;DR
```sh
go get {{ module_name }}@{{ required_version }}
mage vendor
make notice
```
Run `go get {{ module_name }}@{{ required_version }}` in the repository. This adds the module to the requirements list in `go.mod`. In order to add the dependency to the vendor folder, run `mage vendor`. If the dependency contains files which are not copied by this command (e.g. C source files), add them to this list: https://github.com/elastic/beats/tree/master/dev-tools/mage/gomod.go
#### How can I upgrade a dependency?
The process is similar to adding a new dependency. Except for one step. If the dependency is updated to a newer major version, make sure to follow up changes in the root import paths of those libs. I suggest using the tool named `mod` to upgrade it: https://github.com/marwan-at-work/mod
#### How can I use a fork of a dependency?
If you need to use a fork of a dependency, add it to `go.mod` either manually or by running the following command:
```sh
go mod edit -replace {{ dependency_name }}={{ fork_name }}@{{ fork_version }}
```
#### Why is the major version part of the import path?
In a nutshell, it is needed to tell apart major version changes. Their import paths are different because those are incompatible. See more: https://blog.golang.org/v2-go-modules
Thus, when we are releasing v8.0.0, we need to change the import path again. The tool named mod is going to be useful for us in this case, too.
#### Why is `make check` failing?
Are you seeing the following error?
```
make[1]: Leaving directory `/home/travis/gopath/src/github.com/elastic/beats'
diff --git a/go.mod b/go.mod
index 170f6660..fc5fb6ee 100644
--- a/go.mod
+++ b/go.mod
@@ -53,6 +53,7 @@ require (
github.com/dop251/goja_nodejs v0.0.0-20171011081505-adff31b136e6
github.com/dustin/go-humanize v0.0.0-20171111073723-bb3d318650d4
github.com/eclipse/paho.mqtt.golang v1.2.1-0.20200121105743-0d940dd29fd2
+ github.com/elastic/beats v7.6.0+incompatible
github.com/elastic/ecs v1.4.0
github.com/elastic/go-libaudit v0.4.0
github.com/elastic/go-licenser v0.2.1
```
If your `go.mod` lists `github.com/elastic/beats` as a requirement, the repo contains an outdated root import path. Change `github.com/elastic/beats` to `github.com/elastic/beats/v7`.
#### What is next?
After this PR is merged, please rebase all of your open PRs. In most cases rebasing will lead to conflicts. Possible sources of conflicts:
1. New import path conflicts with the old. To resolve it switch every `github.com/elastic/beats` import to `github.com/elasitc/beats/v7`.
2. A new dependency has been added or existing was updated. Update the dependency using go modules.
### Further reading for the interested
- [Golang blog: Using Go modules](https://blog.golang.org/using-go-modules)
- [Golang wiki](https://github.com/golang/go/wiki/Modules)
### Related issues
#15868
kvch
added a commit
to kvch/beats
that referenced
this issue
Mar 3, 2020
This PR introduces the new dependency management system for beats. From now on we are using `go mod` instead of `govendor`. The name of the module we provide is named `github.com/elastic/beats/v7`.
We adopted go modules, but we still keep dependencies under the folder `vendor`. However, it is maintained by `go mod` now by running `mage vendor`. Thus, it is not possible to add local changes to the dependencies there because next time someone runs the command it will be overwritten. **If you need to apply patches to dependencies either fork the repository or try to contribute it back to the original repo.**
This PR does not address the changes in Beat generators. Those are going to be updated in a follow-up PR.
TL;DR
```sh
go get {{ module_name }}@{{ required_version }}
mage vendor
make notice
```
Run `go get {{ module_name }}@{{ required_version }}` in the repository. This adds the module to the requirements list in `go.mod`. In order to add the dependency to the vendor folder, run `mage vendor`. If the dependency contains files which are not copied by this command (e.g. C source files), add them to this list: https://github.com/elastic/beats/tree/master/dev-tools/mage/gomod.go
The process is similar to adding a new dependency. Except for one step. If the dependency is updated to a newer major version, make sure to follow up changes in the root import paths of those libs. I suggest using the tool named `mod` to upgrade it: https://github.com/marwan-at-work/mod
If you need to use a fork of a dependency, add it to `go.mod` either manually or by running the following command:
```sh
go mod edit -replace {{ dependency_name }}={{ fork_name }}@{{ fork_version }}
```
In a nutshell, it is needed to tell apart major version changes. Their import paths are different because those are incompatible. See more: https://blog.golang.org/v2-go-modules
Thus, when we are releasing v8.0.0, we need to change the import path again. The tool named mod is going to be useful for us in this case, too.
Are you seeing the following error?
```
make[1]: Leaving directory `/home/travis/gopath/src/github.com/elastic/beats'
diff --git a/go.mod b/go.mod
index 170f6660..fc5fb6ee 100644
--- a/go.mod
+++ b/go.mod
@@ -53,6 +53,7 @@ require (
github.com/dop251/goja_nodejs v0.0.0-20171011081505-adff31b136e6
github.com/dustin/go-humanize v0.0.0-20171111073723-bb3d318650d4
github.com/eclipse/paho.mqtt.golang v1.2.1-0.20200121105743-0d940dd29fd2
+ github.com/elastic/beats v7.6.0+incompatible
github.com/elastic/ecs v1.4.0
github.com/elastic/go-libaudit v0.4.0
github.com/elastic/go-licenser v0.2.1
```
If your `go.mod` lists `github.com/elastic/beats` as a requirement, the repo contains an outdated root import path. Change `github.com/elastic/beats` to `github.com/elastic/beats/v7`.
After this PR is merged, please rebase all of your open PRs. In most cases rebasing will lead to conflicts. Possible sources of conflicts:
1. New import path conflicts with the old. To resolve it switch every `github.com/elastic/beats` import to `github.com/elasitc/beats/v7`.
2. A new dependency has been added or existing was updated. Update the dependency using go modules.
- [Golang blog: Using Go modules](https://blog.golang.org/using-go-modules)
- [Golang wiki](https://github.com/golang/go/wiki/Modules)
elastic#15868
(cherry picked from commit 95626b8)
kvch
added a commit
to kvch/beats
that referenced
this issue
Mar 3, 2020
This PR introduces the new dependency management system for beats. From now on we are using `go mod` instead of `govendor`. The name of the module we provide is named `github.com/elastic/beats/v7`.
We adopted go modules, but we still keep dependencies under the folder `vendor`. However, it is maintained by `go mod` now by running `mage vendor`. Thus, it is not possible to add local changes to the dependencies there because next time someone runs the command it will be overwritten. **If you need to apply patches to dependencies either fork the repository or try to contribute it back to the original repo.**
This PR does not address the changes in Beat generators. Those are going to be updated in a follow-up PR.
TL;DR
```sh
go get {{ module_name }}@{{ required_version }}
mage vendor
make notice
```
Run `go get {{ module_name }}@{{ required_version }}` in the repository. This adds the module to the requirements list in `go.mod`. In order to add the dependency to the vendor folder, run `mage vendor`. If the dependency contains files which are not copied by this command (e.g. C source files), add them to this list: https://github.com/elastic/beats/tree/master/dev-tools/mage/gomod.go
The process is similar to adding a new dependency. Except for one step. If the dependency is updated to a newer major version, make sure to follow up changes in the root import paths of those libs. I suggest using the tool named `mod` to upgrade it: https://github.com/marwan-at-work/mod
If you need to use a fork of a dependency, add it to `go.mod` either manually or by running the following command:
```sh
go mod edit -replace {{ dependency_name }}={{ fork_name }}@{{ fork_version }}
```
In a nutshell, it is needed to tell apart major version changes. Their import paths are different because those are incompatible. See more: https://blog.golang.org/v2-go-modules
Thus, when we are releasing v8.0.0, we need to change the import path again. The tool named mod is going to be useful for us in this case, too.
Are you seeing the following error?
```
make[1]: Leaving directory `/home/travis/gopath/src/github.com/elastic/beats'
diff --git a/go.mod b/go.mod
index 170f6660..fc5fb6ee 100644
--- a/go.mod
+++ b/go.mod
@@ -53,6 +53,7 @@ require (
github.com/dop251/goja_nodejs v0.0.0-20171011081505-adff31b136e6
github.com/dustin/go-humanize v0.0.0-20171111073723-bb3d318650d4
github.com/eclipse/paho.mqtt.golang v1.2.1-0.20200121105743-0d940dd29fd2
+ github.com/elastic/beats v7.6.0+incompatible
github.com/elastic/ecs v1.4.0
github.com/elastic/go-libaudit v0.4.0
github.com/elastic/go-licenser v0.2.1
```
If your `go.mod` lists `github.com/elastic/beats` as a requirement, the repo contains an outdated root import path. Change `github.com/elastic/beats` to `github.com/elastic/beats/v7`.
After this PR is merged, please rebase all of your open PRs. In most cases rebasing will lead to conflicts. Possible sources of conflicts:
1. New import path conflicts with the old. To resolve it switch every `github.com/elastic/beats` import to `github.com/elasitc/beats/v7`.
2. A new dependency has been added or existing was updated. Update the dependency using go modules.
- [Golang blog: Using Go modules](https://blog.golang.org/using-go-modules)
- [Golang wiki](https://github.com/golang/go/wiki/Modules)
elastic#15868
(cherry picked from commit 95626b8)
kvch
added a commit
that referenced
this issue
Mar 3, 2020
* Go modules (#15853) This PR introduces the new dependency management system for beats. From now on we are using `go mod` instead of `govendor`. The name of the module we provide is named `github.com/elastic/beats/v7`. We adopted go modules, but we still keep dependencies under the folder `vendor`. However, it is maintained by `go mod` now by running `mage vendor`. Thus, it is not possible to add local changes to the dependencies there because next time someone runs the command it will be overwritten. **If you need to apply patches to dependencies either fork the repository or try to contribute it back to the original repo.** This PR does not address the changes in Beat generators. Those are going to be updated in a follow-up PR. TL;DR ```sh go get {{ module_name }}@{{ required_version }} mage vendor make notice ``` Run `go get {{ module_name }}@{{ required_version }}` in the repository. This adds the module to the requirements list in `go.mod`. In order to add the dependency to the vendor folder, run `mage vendor`. If the dependency contains files which are not copied by this command (e.g. C source files), add them to this list: https://github.com/elastic/beats/tree/master/dev-tools/mage/gomod.go The process is similar to adding a new dependency. Except for one step. If the dependency is updated to a newer major version, make sure to follow up changes in the root import paths of those libs. I suggest using the tool named `mod` to upgrade it: https://github.com/marwan-at-work/mod If you need to use a fork of a dependency, add it to `go.mod` either manually or by running the following command: ```sh go mod edit -replace {{ dependency_name }}={{ fork_name }}@{{ fork_version }} ``` In a nutshell, it is needed to tell apart major version changes. Their import paths are different because those are incompatible. See more: https://blog.golang.org/v2-go-modules Thus, when we are releasing v8.0.0, we need to change the import path again. The tool named mod is going to be useful for us in this case, too. Are you seeing the following error? ``` make[1]: Leaving directory `/home/travis/gopath/src/github.com/elastic/beats' diff --git a/go.mod b/go.mod index 170f6660..fc5fb6ee 100644 --- a/go.mod +++ b/go.mod @@ -53,6 +53,7 @@ require ( github.com/dop251/goja_nodejs v0.0.0-20171011081505-adff31b136e6 github.com/dustin/go-humanize v0.0.0-20171111073723-bb3d318650d4 github.com/eclipse/paho.mqtt.golang v1.2.1-0.20200121105743-0d940dd29fd2 + github.com/elastic/beats v7.6.0+incompatible github.com/elastic/ecs v1.4.0 github.com/elastic/go-libaudit v0.4.0 github.com/elastic/go-licenser v0.2.1 ``` If your `go.mod` lists `github.com/elastic/beats` as a requirement, the repo contains an outdated root import path. Change `github.com/elastic/beats` to `github.com/elastic/beats/v7`. After this PR is merged, please rebase all of your open PRs. In most cases rebasing will lead to conflicts. Possible sources of conflicts: 1. New import path conflicts with the old. To resolve it switch every `github.com/elastic/beats` import to `github.com/elasitc/beats/v7`. 2. A new dependency has been added or existing was updated. Update the dependency using go modules. - [Golang blog: Using Go modules](https://blog.golang.org/using-go-modules) - [Golang wiki](https://github.com/golang/go/wiki/Modules) #15868 (cherry picked from commit 95626b8) * update generated file
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Go modules is going to be the dependency management system from Go 1.14. To enable a seamless update from the current
.go-version(1.13.x), we should adopt the system before 1.14 is released.TODO
add minimal version requirements Initialize go.mod from vendor #15688
generate
NOTICE.txtfrommodules.txtInitialize go.mod from vendor #15688add tools (goimports, mage) and install it from vendor folder Install tools (mage, goimports, etc.) from vendor folder #15998
add dependencies with non v1 major versions Add dependencies with v2 or bigger major versions #16100
find version information for dependencies which does not have any Go modules support #15868 (comment)
add C files to
vendor(e.g.go-daemon,godror) Add C dependencies to vendor #16127use our for of
go-plugins-helpersVarious fixes: use fork of go-plugins-helpers, Functionbeat, templates #16191follow-up changes in the CIs (
Jenkinsfile,.travis.yml) Adjust CI configuration #16216migrate beat generators to go modules Fix generator jobs for go modules #16288 Copy missing go version to the generated Beats #17105
include major version in import path in beats Add major version "v7" to import path #16331
build from local vendor folder Add C dependencies to vendor #16127 Add GOFLAGS to system-test-env #16236
build beats inside
$GOPATHBuild beats outside GOPATH #16329build beats outside
$GOPATHBuild beats outside GOPATH #16329package beats inside
$GOPATHBuild beats outside GOPATH #16329package beats outside
$GOPATHBuild beats outside GOPATH #16329pass on all automated tests
knowledge share
run beats-tester
The text was updated successfully, but these errors were encountered: