Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fork & Bug: API & SDK are not tested #61

Closed
cipherboy opened this issue Jan 25, 2024 · 7 comments · Fixed by #155
Closed

Fork & Bug: API & SDK are not tested #61

cipherboy opened this issue Jan 25, 2024 · 7 comments · Fixed by #155
Milestone

Comments

@cipherboy
Copy link
Member

OpenBao has a few Go modules in the same repository:

./api/auth/approle/go.mod
./api/auth/kubernetes/go.mod
./api/auth/ldap/go.mod
./api/auth/userpass/go.mod
./api/go.mod
./sdk/go.mod
./vault/hcp_link/proto/go.mod
./go.mod

However, when running go list ./... from the root, a few aren't found:

Output
$ go list ./...
github.com/openbao/openbao
github.com/openbao/openbao/audit
github.com/openbao/openbao/builtin/audit/file
github.com/openbao/openbao/builtin/audit/socket
github.com/openbao/openbao/builtin/audit/syslog
github.com/openbao/openbao/builtin/credential/approle
github.com/openbao/openbao/builtin/credential/approle/cmd/approle
github.com/openbao/openbao/builtin/credential/cert
github.com/openbao/openbao/builtin/credential/cert/cmd/cert
github.com/openbao/openbao/builtin/credential/jwt
github.com/openbao/openbao/builtin/credential/jwt/cmd/jwt
github.com/openbao/openbao/builtin/credential/kerberos
github.com/openbao/openbao/builtin/credential/kerberos/cmd/kerberos
github.com/openbao/openbao/builtin/credential/kerberos/cmd/login-kerb
github.com/openbao/openbao/builtin/credential/kubernetes
github.com/openbao/openbao/builtin/credential/kubernetes/cmd/kubernetes
github.com/openbao/openbao/builtin/credential/ldap
github.com/openbao/openbao/builtin/credential/ldap/cmd/ldap
github.com/openbao/openbao/builtin/credential/radius
github.com/openbao/openbao/builtin/credential/radius/cmd/radius
github.com/openbao/openbao/builtin/credential/token
github.com/openbao/openbao/builtin/credential/userpass
github.com/openbao/openbao/builtin/credential/userpass/cmd/userpass
github.com/openbao/openbao/builtin/logical/database
github.com/openbao/openbao/builtin/logical/database/dbplugin
github.com/openbao/openbao/builtin/logical/kubernetes
github.com/openbao/openbao/builtin/logical/kubernetes/cmd/kubernetes
github.com/openbao/openbao/builtin/logical/kubernetes/integrationtest
github.com/openbao/openbao/builtin/logical/kv
github.com/openbao/openbao/builtin/logical/kv/cmd/kv
github.com/openbao/openbao/builtin/logical/openldap
github.com/openbao/openbao/builtin/logical/openldap/client
github.com/openbao/openbao/builtin/logical/openldap/client/tools
github.com/openbao/openbao/builtin/logical/openldap/cmd/openldap
github.com/openbao/openbao/builtin/logical/openldap/ldapifc
github.com/openbao/openbao/builtin/logical/pki
github.com/openbao/openbao/builtin/logical/pki/cmd/pki
github.com/openbao/openbao/builtin/logical/pki/dnstest
github.com/openbao/openbao/builtin/logical/pkiext
github.com/openbao/openbao/builtin/logical/pkiext/pkiext_binary
github.com/openbao/openbao/builtin/logical/rabbitmq
github.com/openbao/openbao/builtin/logical/rabbitmq/cmd/rabbitmq
github.com/openbao/openbao/builtin/logical/ssh
github.com/openbao/openbao/builtin/logical/ssh/cmd/ssh
github.com/openbao/openbao/builtin/logical/totp
github.com/openbao/openbao/builtin/logical/totp/cmd/totp
github.com/openbao/openbao/builtin/logical/transit
github.com/openbao/openbao/builtin/logical/transit/cmd/transit
github.com/openbao/openbao/builtin/plugin
github.com/openbao/openbao/builtin/plugin/v5
github.com/openbao/openbao/command
github.com/openbao/openbao/command/agent
github.com/openbao/openbao/command/agent/config
github.com/openbao/openbao/command/agent/exec
github.com/openbao/openbao/command/agent/exec/test-app
github.com/openbao/openbao/command/agent/internal/ctmanager
github.com/openbao/openbao/command/agent/template
github.com/openbao/openbao/command/agentproxyshared
github.com/openbao/openbao/command/agentproxyshared/auth
github.com/openbao/openbao/command/agentproxyshared/auth/approle
github.com/openbao/openbao/command/agentproxyshared/auth/cert
github.com/openbao/openbao/command/agentproxyshared/auth/jwt
github.com/openbao/openbao/command/agentproxyshared/auth/kerberos
github.com/openbao/openbao/command/agentproxyshared/auth/kubernetes
github.com/openbao/openbao/command/agentproxyshared/auth/token-file
github.com/openbao/openbao/command/agentproxyshared/cache
github.com/openbao/openbao/command/agentproxyshared/cache/cacheboltdb
github.com/openbao/openbao/command/agentproxyshared/cache/cachememdb
github.com/openbao/openbao/command/agentproxyshared/cache/keymanager
github.com/openbao/openbao/command/agentproxyshared/sink
github.com/openbao/openbao/command/agentproxyshared/sink/file
github.com/openbao/openbao/command/agentproxyshared/sink/inmem
github.com/openbao/openbao/command/agentproxyshared/sink/mock
github.com/openbao/openbao/command/agentproxyshared/winsvc
github.com/openbao/openbao/command/config
github.com/openbao/openbao/command/healthcheck
github.com/openbao/openbao/command/proxy/config
github.com/openbao/openbao/command/server
github.com/openbao/openbao/command/test-backend
github.com/openbao/openbao/command/token
github.com/openbao/openbao/helper/benchhelpers
github.com/openbao/openbao/helper/builtinplugins
github.com/openbao/openbao/helper/constants
github.com/openbao/openbao/helper/dhutil
github.com/openbao/openbao/helper/experiments
github.com/openbao/openbao/helper/fairshare
github.com/openbao/openbao/helper/flag-kv
github.com/openbao/openbao/helper/flag-slice
github.com/openbao/openbao/helper/forwarding
github.com/openbao/openbao/helper/hostutil
github.com/openbao/openbao/helper/identity
github.com/openbao/openbao/helper/identity/mfa
github.com/openbao/openbao/helper/locking
github.com/openbao/openbao/helper/logging
github.com/openbao/openbao/helper/metricsutil
github.com/openbao/openbao/helper/monitor
github.com/openbao/openbao/helper/namespace
github.com/openbao/openbao/helper/osutil
github.com/openbao/openbao/helper/parseip
github.com/openbao/openbao/helper/pgpkeys
github.com/openbao/openbao/helper/policies
github.com/openbao/openbao/helper/proxyutil
github.com/openbao/openbao/helper/random
github.com/openbao/openbao/helper/storagepacker
github.com/openbao/openbao/helper/testhelpers
github.com/openbao/openbao/helper/testhelpers/cassandra
github.com/openbao/openbao/helper/testhelpers/certhelpers
github.com/openbao/openbao/helper/testhelpers/consul
github.com/openbao/openbao/helper/testhelpers/corehelpers
github.com/openbao/openbao/helper/testhelpers/ldap
github.com/openbao/openbao/helper/testhelpers/logical
github.com/openbao/openbao/helper/testhelpers/mysql
github.com/openbao/openbao/helper/testhelpers/pluginhelpers
github.com/openbao/openbao/helper/testhelpers/postgresql
github.com/openbao/openbao/helper/testhelpers/seal
github.com/openbao/openbao/helper/testhelpers/teststorage
github.com/openbao/openbao/helper/timeutil
github.com/openbao/openbao/helper/useragent
github.com/openbao/openbao/helper/versions
github.com/openbao/openbao/http
github.com/openbao/openbao/internal
github.com/openbao/openbao/internalshared/configutil
github.com/openbao/openbao/internalshared/listenerutil
github.com/openbao/openbao/physical/raft
github.com/openbao/openbao/plugins/database/cassandra
github.com/openbao/openbao/plugins/database/cassandra/cassandra-database-plugin
github.com/openbao/openbao/plugins/database/influxdb
github.com/openbao/openbao/plugins/database/influxdb/influxdb-database-plugin
github.com/openbao/openbao/plugins/database/mysql
github.com/openbao/openbao/plugins/database/mysql/mysql-database-plugin
github.com/openbao/openbao/plugins/database/mysql/mysql-legacy-database-plugin
github.com/openbao/openbao/plugins/database/postgresql
github.com/openbao/openbao/plugins/database/postgresql/postgresql-database-plugin
github.com/openbao/openbao/plugins/database/postgresql/scram
github.com/openbao/openbao/plugins/database/redis
github.com/openbao/openbao/plugins/database/redis/redis-database-plugin/vault-plugin-database-redis
github.com/openbao/openbao/serviceregistration
github.com/openbao/openbao/serviceregistration/consul
github.com/openbao/openbao/serviceregistration/kubernetes
github.com/openbao/openbao/serviceregistration/kubernetes/client
github.com/openbao/openbao/serviceregistration/kubernetes/client/cmd/kubeclient
github.com/openbao/openbao/serviceregistration/kubernetes/testing
github.com/openbao/openbao/shamir
github.com/openbao/openbao/tools/codechecker
github.com/openbao/openbao/tools/codechecker/pkg/godoctests
github.com/openbao/openbao/tools/codechecker/pkg/gonilnilfunctions
github.com/openbao/openbao/vault
github.com/openbao/openbao/vault/activity
github.com/openbao/openbao/vault/activity/generation
github.com/openbao/openbao/vault/cluster
github.com/openbao/openbao/vault/diagnose
github.com/openbao/openbao/vault/eventbus
github.com/openbao/openbao/vault/external_tests/api
github.com/openbao/openbao/vault/external_tests/approle
github.com/openbao/openbao/vault/external_tests/expiration
github.com/openbao/openbao/vault/external_tests/hcp_link
github.com/openbao/openbao/vault/external_tests/identity
github.com/openbao/openbao/vault/external_tests/kv
github.com/openbao/openbao/vault/external_tests/metrics
github.com/openbao/openbao/vault/external_tests/mfa
github.com/openbao/openbao/vault/external_tests/misc
github.com/openbao/openbao/vault/external_tests/misc/misc_binary
github.com/openbao/openbao/vault/external_tests/plugin
github.com/openbao/openbao/vault/external_tests/policy
github.com/openbao/openbao/vault/external_tests/pprof
github.com/openbao/openbao/vault/external_tests/pprof/pprof_binary
github.com/openbao/openbao/vault/external_tests/quotas
github.com/openbao/openbao/vault/external_tests/raft
github.com/openbao/openbao/vault/external_tests/raft/raft_binary
github.com/openbao/openbao/vault/external_tests/raftha
github.com/openbao/openbao/vault/external_tests/replication_binary
github.com/openbao/openbao/vault/external_tests/response
github.com/openbao/openbao/vault/external_tests/router
github.com/openbao/openbao/vault/external_tests/sealmigration
github.com/openbao/openbao/vault/external_tests/sealmigrationext
github.com/openbao/openbao/vault/external_tests/token
github.com/openbao/openbao/vault/hcp_link
github.com/openbao/openbao/vault/hcp_link/capabilities
github.com/openbao/openbao/vault/hcp_link/capabilities/api_capability
github.com/openbao/openbao/vault/hcp_link/capabilities/link_control
github.com/openbao/openbao/vault/hcp_link/capabilities/meta
github.com/openbao/openbao/vault/hcp_link/capabilities/node_status
github.com/openbao/openbao/vault/hcp_link/internal
github.com/openbao/openbao/vault/quotas
github.com/openbao/openbao/vault/replication
github.com/openbao/openbao/vault/seal
github.com/openbao/openbao/vault/tokens
github.com/openbao/openbao/version

In particular, sdk and api aren't listed. This issue appears to persist on the original release/1.14.x branch. The result is that both have test failures.

This usage of go list ./... and omission of these modules needs to be addressed in the Makefile, building, and CI systems at least, in addition to fixing the test failures.

@pdecat
Copy link

pdecat commented Jan 26, 2024

I guess using go list ./... was essentially needed when dependencies were stored in /vendor which is no longer the case since the switch to go modules. Not sure about the need to exclude /integ though.

Also, this prevents getting test output live as I've discovered some time ago in hashicorp/terraform-provider-helm#186 (review) (maybe not true any longer nowadays).

@cipherboy
Copy link
Member Author

cipherboy commented Jan 26, 2024

I guess using go list ./... was essentially needed when dependencies were stored in /vendor which is no longer the case since the switch to go modules. Not sure about the need to exclude /integ though.

/integ doesn't seem to exist in this project, so it is probably safe to remove it from the exclusion.

Also, this prevents getting test output live as I've discovered some time ago in hashicorp/terraform-provider-helm#186 (review) (maybe not true any longer nowadays).

Just so I'm clear, what is the alternative you're proposing? Note that this repo has both sub-modules (/go.mod, /api/go.mod, and /sdk/go.mod, plus a few other smaller ones) and sub-packages within the main and nested modules (/builtin/logical/pki under /go.mod, /sdk/helper/certutil under /sdk/go.mod, ...).

From my trial, these don't work:

openbao]$ go test github.com/openbao/openbao
ok  	github.com/openbao/openbao	0.021s [no tests to run]
openbao]$ go test 
testing: warning: no tests to run
PASS
ok  	github.com/openbao/openbao	0.021s
openbao]$ go test .
ok  	github.com/openbao/openbao	(cached) [no tests to run]
openbao]$ go test openbao
package openbao is not in std (/usr/local/go/src/openbao)

And I get a rather cryptic errors trying to do things like:

openbao]$ go test ./sdk/
no Go files in /home/cipherboy/GitHub/openbao/openbao/sdk
openbao]$ go test ./sdk/helper/
no Go files in /home/cipherboy/GitHub/openbao/openbao/sdk/helper
openbao]$ go test ./sdk/helper/certutil/
main module (github.com/openbao/openbao) does not contain package github.com/openbao/openbao/sdk/helper/certutil
openbao]$ go test sdk/helper/certutil/
package sdk/helper/certutil is not in std (/usr/local/go/src/sdk/helper/certutil)
openbao]$ go test ./sdk/helper/certutil
main module (github.com/openbao/openbao) does not contain package github.com/openbao/openbao/sdk/helper/certutil
openbao]$ go test github.com/openbao/openbao/sdk
no required module provides package github.com/openbao/openbao/sdk; to add it:
	go get github.com/openbao/openbao/sdk

though interestingly, cd'ing is sufficient:

openbao]$ cd sdk
sdk]$ go test helper/certutil
package helper/certutil is not in std (/usr/local/go/src/helper/certutil)
sdk]$ go test github.com/openbao/openbao/sdk/helper/certutil
ok  	github.com/openbao/openbao/sdk/helper/certutil	(cached)
sdk]$ go test -count=1 github.com/openbao/openbao/sdk/helper/certutil
ok  	github.com/openbao/openbao/sdk/helper/certutil	0.924s
sdk]$ cd ..
openbao]$ go test -count=1 github.com/openbao/openbao/sdk/helper/certutil
ok  	github.com/openbao/openbao/sdk/helper/certutil	0.924s

Even though I'm able to do api from the root:

openbao]$ go test ./api/
main module (github.com/openbao/openbao) does not contain package github.com/openbao/openbao/api
openbao]$ go test api
package api is not in std (/usr/local/go/src/api)
openbao]$ go test github.com/openbao/openbao/api
ok  	github.com/openbao/openbao/api	13.452s

So I think the best thing to do is something like:

[cipherboy@lp10 openbao]$ go test $(find -iname go.mod | sed 's/^./github.com\/openbao\/openbao/g' | sed 's/go\.mod$/.../g')
go: warning: "github.com/openbao/openbao/api/auth/kubernetes/..." matched no packages
go: warning: "github.com/openbao/openbao/api/auth/ldap/..." matched no packages
go: warning: "github.com/openbao/openbao/builtin/credential/jwt/tools/..." matched no packages
go: warning: "github.com/openbao/openbao/builtin/credential/kubernetes/integrationtest/..." matched no packages
go: warning: "github.com/openbao/openbao/builtin/logical/openldap/tools/..." matched no packages
go: warning: "github.com/openbao/openbao/plugins/database/redis/tools/..." matched no packages
ok  	github.com/openbao/openbao/api/auth/approle	0.007s
ok  	github.com/openbao/openbao/api/auth/userpass	0.007s
?   	github.com/openbao/openbao/sdk/database/dbplugin	[no test files]
ok  	github.com/openbao/openbao/api	(cached)
?   	github.com/openbao/openbao/sdk/database/dbplugin/v5/proto	[no test files]
?   	github.com/openbao/openbao/sdk/database/dbplugin/v5/testing	[no test files]
ok  	github.com/openbao/openbao/sdk/database/dbplugin/v5	0.009s
ok  	github.com/openbao/openbao/sdk/database/helper/connutil	0.006s
... lots and lots and lots of output trimmed ...

...perhaps?

@pdecat
Copy link

pdecat commented Jan 26, 2024

Hmm, I can't even build the development branch right now:

go build
# github.com/openbao/openbao/helper/testhelpers/teststorage
helper/testhelpers/teststorage/teststorage.go:264:10: cannot use logicalKv.Factory (value of type func(ctx "context".Context, conf *"github.com/hashicorp/vault/sdk/logical".BackendConfig) ("github.com/hashicorp/vault/sdk/logical".Backend, error)) as "github.com/openbao/openbao/sdk/logical".Factory value in map literal
# github.com/openbao/openbao/helper/builtinplugins
helper/builtinplugins/registry.go:102:26: cannot use credAliCloud.Factory (value of type func(ctx "context".Context, conf *"github.com/hashicorp/vault/sdk/logical".BackendConfig) ("github.com/hashicorp/vault/sdk/logical".Backend, error)) as "github.com/openbao/openbao/sdk/logical".Factory value in struct literal
helper/builtinplugins/registry.go:109:28: cannot use credAzure.Factory (value of type func(ctx "context".Context, c *"github.com/hashicorp/vault/sdk/logical".BackendConfig) ("github.com/hashicorp/vault/sdk/logical".Backend, error)) as "github.com/openbao/openbao/sdk/logical".Factory value in struct literal
helper/builtinplugins/registry.go:110:28: cannot use credCentrify.Factory (value of type func(ctx "context".Context, conf *"github.com/hashicorp/vault/sdk/logical".BackendConfig) ("github.com/hashicorp/vault/sdk/logical".Backend, error)) as "github.com/openbao/openbao/sdk/logical".Factory value in struct literal
helper/builtinplugins/registry.go:112:28: cannot use credCF.Factory (value of type func(ctx "context".Context, conf *"github.com/hashicorp/vault/sdk/logical".BackendConfig) ("github.com/hashicorp/vault/sdk/logical".Backend, error)) as "github.com/openbao/openbao/sdk/logical".Factory value in struct literal
helper/builtinplugins/registry.go:113:28: cannot use credGcp.Factory (value of type func(ctx "context".Context, conf *"github.com/hashicorp/vault/sdk/logical".BackendConfig) ("github.com/hashicorp/vault/sdk/logical".Backend, error)) as "github.com/openbao/openbao/sdk/logical".Factory value in struct literal
helper/builtinplugins/registry.go:115:28: cannot use credJWT.Factory (value of type func(ctx "context".Context, c *"github.com/hashicorp/vault/sdk/logical".BackendConfig) ("github.com/hashicorp/vault/sdk/logical".Backend, error)) as "github.com/openbao/openbao/sdk/logical".Factory value in struct literal
helper/builtinplugins/registry.go:116:28: cannot use credKerb.Factory (value of type func(ctx "context".Context, c *"github.com/hashicorp/vault/sdk/logical".BackendConfig) ("github.com/hashicorp/vault/sdk/logical".Backend, error)) as "github.com/openbao/openbao/sdk/logical".Factory value in struct literal
helper/builtinplugins/registry.go:117:28: cannot use credKube.Factory (value of type func(ctx "context".Context, conf *"github.com/hashicorp/vault/sdk/logical".BackendConfig) ("github.com/hashicorp/vault/sdk/logical".Backend, error)) as "github.com/openbao/openbao/sdk/logical".Factory value in struct literal
helper/builtinplugins/registry.go:119:28: cannot use credOCI.Factory (value of type func(ctx "context".Context, conf *"github.com/hashicorp/vault/sdk/logical".BackendConfig) ("github.com/hashicorp/vault/sdk/logical".Backend, error)) as "github.com/openbao/openbao/sdk/logical".Factory value in struct literal
helper/builtinplugins/registry.go:120:28: cannot use credJWT.Factory (value of type func(ctx "context".Context, c *"github.com/hashicorp/vault/sdk/logical".BackendConfig) ("github.com/hashicorp/vault/sdk/logical".Backend, error)) as "github.com/openbao/openbao/sdk/logical".Factory value in struct literal
helper/builtinplugins/registry.go:120:28: too many errors

Do you have additional changes locally to make it work?

@cipherboy
Copy link
Member Author

cipherboy commented Jan 26, 2024

Hi @pdecat sorry, this was discussed on the LF Element/Matrix bridge but not brought here -- development won't build presently until we get some consensus around #55 and #64 (to be able to decide if we're pruning any plugins, and how we're maintaining built-in plugins: part of the repo or externally).

In the mean time, aligning with #64, PR #56 has a branch that should build but fail tests, and my cipherboy/openbao:pruned-hcp-link-too branch should have more progress towards passing tests (past what's in that proposal), including some changes that have been merged into development after starting that branch (for some reason, git subtrees aren't rebasing nicely, even with ----rebase-merges that I haven't looked into yet -- most of the changes I'll open as separate PRs and cherry-pick the non-subtree changes into relevant PRs).

@pdecat
Copy link

pdecat commented Jan 26, 2024

Just so I'm clear, what is the alternative you're proposing?

I was essentially proposing to run go test ./... in each module root, maybe using cd as you mentioned.

But right now, I see this is triggering lots of errors even for the root module:

# cd ~/go/src/github.com/openbao/openbao
# go test ./...
?       github.com/openbao/openbao/builtin/audit/socket [no test files]
?       github.com/openbao/openbao/builtin/audit/syslog [no test files]
ok      github.com/openbao/openbao      (cached) [no tests to run]
ok      github.com/openbao/openbao/audit        (cached)
ok      github.com/openbao/openbao/builtin/audit/file   (cached)
?       github.com/openbao/openbao/builtin/credential/approle/cmd/approle       [no test files]
?       github.com/openbao/openbao/builtin/credential/aws/cmd/aws       [no test files]
ok      github.com/openbao/openbao/builtin/credential/approle   (cached)
?       github.com/openbao/openbao/builtin/credential/cert/cmd/cert     [no test files]
ok      github.com/openbao/openbao/builtin/credential/aws       (cached)
ok      github.com/openbao/openbao/builtin/credential/aws/pkcs7 (cached)
ok      github.com/openbao/openbao/builtin/credential/cert      (cached)
ok      github.com/openbao/openbao/builtin/credential/github    (cached)
?       github.com/openbao/openbao/builtin/credential/github/cmd/github [no test files]
?       github.com/openbao/openbao/builtin/credential/jwt/cmd/jwt       [no test files]
?       github.com/openbao/openbao/builtin/credential/kerberos/cmd/kerberos     [no test files]
ok      github.com/openbao/openbao/builtin/credential/jwt       (cached)
?       github.com/openbao/openbao/builtin/credential/kubernetes/cmd/kubernetes [no test files]
ok      github.com/openbao/openbao/builtin/credential/kerberos  (cached)
?       github.com/openbao/openbao/builtin/credential/kerberos/cmd/login-kerb   [no test files]
ok      github.com/openbao/openbao/builtin/credential/kubernetes        (cached)
?       github.com/openbao/openbao/builtin/credential/ldap/cmd/ldap     [no test files]
?       github.com/openbao/openbao/builtin/credential/okta/cmd/okta     [no test files]
ok      github.com/openbao/openbao/builtin/credential/ldap      (cached)
ok      github.com/openbao/openbao/builtin/credential/okta      (cached)
ok      github.com/openbao/openbao/builtin/credential/radius    (cached)
# github.com/openbao/openbao/builtin/credential/userpass [github.com/openbao/openbao/builtin/credential/userpass.test]
builtin/credential/userpass/stepwise_test.go:22:20: cannot use api.PluginTypeCredential (constant 1 of type "github.com/openbao/openbao/api".PluginType) as "github.com/hashicorp/vault/api".PluginType value in struct literal
builtin/credential/userpass/stepwise_test.go:61:11: cannot use func(resp *api.Secret, err error) error {…} (value of type func(resp *"github.com/openbao/openbao/api".Secret, err error) error) as stepwise.AssertionFunc value in struct literal
# github.com/openbao/openbao/builtin/logical/kubernetes/cmd/kubernetes
builtin/logical/kubernetes/cmd/kubernetes/main.go:24:23: cannot use kubesecrets.Factory (value of type func(ctx "context".Context, conf *"github.com/hashicorp/vault/sdk/logical".BackendConfig) ("github.com/hashicorp/vault/sdk/logical".Backend, error)) as "github.com/openbao/openbao/sdk/logical".Factory value in struct literal
?       github.com/openbao/openbao/builtin/credential/radius/cmd/radius [no test files]
?       github.com/openbao/openbao/builtin/credential/token     [no test files]
FAIL    github.com/openbao/openbao/builtin/credential/userpass [build failed]
?       github.com/openbao/openbao/builtin/credential/userpass/cmd/userpass     [no test files]
?       github.com/openbao/openbao/builtin/logical/openldap/client/tools        [no test files]
?       github.com/openbao/openbao/builtin/logical/openldap/cmd/openldap        [no test files]
?       github.com/openbao/openbao/builtin/logical/openldap/ldapifc     [no test files]
?       github.com/openbao/openbao/builtin/logical/pki/cmd/pki  [no test files]
?       github.com/openbao/openbao/builtin/logical/pki/dnstest  [no test files]
# github.com/openbao/openbao/builtin/logical/transit [github.com/openbao/openbao/builtin/logical/transit.test]
builtin/logical/transit/stepwise_test.go:24:20: cannot use api.PluginTypeSecrets (constant 3 of type "github.com/openbao/openbao/api".PluginType) as "github.com/hashicorp/vault/api".PluginType value in struct literal
builtin/logical/transit/stepwise_test.go:62:11: cannot use func(resp *api.Secret, err error) error {…} (value of type func(resp *"github.com/openbao/openbao/api".Secret, err error) error) as stepwise.AssertionFunc value in struct literal
builtin/logical/transit/stepwise_test.go:104:11: cannot use func(resp *api.Secret, err error) error {…} (value of type func(resp *"github.com/openbao/openbao/api".Secret, err error) error) as stepwise.AssertionFunc value in struct literal
builtin/logical/transit/stepwise_test.go:177:11: cannot use func(resp *api.Secret, err error) error {…} (value of type func(resp *"github.com/openbao/openbao/api".Secret, err error) error) as stepwise.AssertionFunc value in struct literal
builtin/logical/transit/stepwise_test.go:201:11: cannot use func(resp *api.Secret, err error) error {…} (value of type func(resp *"github.com/openbao/openbao/api".Secret, err error) error) as stepwise.AssertionFunc value in struct literal
?       github.com/openbao/openbao/builtin/logical/rabbitmq/cmd/rabbitmq        [no test files]
... lots and lots and lots of output trimmed ...

@cipherboy
Copy link
Member Author

cipherboy commented Jan 26, 2024

@pdecat Hmm, I would've expected those failures if you were before 34c0163 on pruned-hcp-link-too, or if you were on plugin-proposal, but not if you were on 959b345, which I think is where cipherboy/openbao:pruned-hcp-link-too currently points to.

Edit: ec4698a is now the latest commit. All the tests should pass except these:

--- FAIL: TestOperatorDiagnoseCommand_Run (0.00s)
    --- FAIL: TestOperatorDiagnoseCommand_Run/validations (0.00s)
        --- FAIL: TestOperatorDiagnoseCommand_Run/validations/diagnose_invalid_https_sr (2.75s)
            operator_diagnose_test.go:491: Did not find expected test results: could not find expected test result: Check Service Discovery
        --- FAIL: TestOperatorDiagnoseCommand_Run/validations/diagnose_direct_storage_access (2.20s)
            operator_diagnose_test.go:491: Did not find expected test results: section Check Storage, status mismatch: warn vs fail, got error Diagnose could not initialize storage backend.
        --- FAIL: TestOperatorDiagnoseCommand_Run/validations/diagnose_ok (2.88s)
            operator_diagnose_test.go:491: Did not find expected test results: could not find expected test result: Start Listeners
        --- FAIL: TestOperatorDiagnoseCommand_Run/validations/diagnose_invalid_https_storage (2.53s)
            operator_diagnose_test.go:491: Did not find expected test results: [ failure ] Create Storage Backend: Unknown storage type consul
        --- FAIL: TestOperatorDiagnoseCommand_Run/validations/diagnose_seal_transit_tls_check_fail (2.64s)
            operator_diagnose_test.go:491: Did not find expected test results: could not find expected test result: Check Transit Seal TLS
        --- FAIL: TestOperatorDiagnoseCommand_Run/validations/diagnose_invalid_https_hastorage (2.58s)
            operator_diagnose_test.go:491: Did not find expected test results: section Check Storage, status mismatch: warn vs fail, got error Diagnose could not initialize storage backend.
        --- FAIL: TestOperatorDiagnoseCommand_Run/validations/diagnose_listener_config_ok (2.61s)
            operator_diagnose_test.go:491: Did not find expected test results: could not find expected test result: Start Listeners

in

FAIL	github.com/openbao/openbao/command	387.652s

These require consul storage, and I've been too lazy to rewrite the HCL files they depend on -- if someone wants to take that on and propose a PR, that'd be appreciated!

@pdecat
Copy link

pdecat commented Jan 27, 2024

Oh ok, I misread your previous comment and checked out the wrong branch with gh pr checkout 56 🤦.

Things are way better on cipherboy/openbao:pruned-hcp-link-too, thanks!

@cipherboy cipherboy changed the title Bug: API & SDK are not tested Fork & Bug: API & SDK are not tested Feb 5, 2024
@naphelps naphelps added this to the GA milestone Feb 5, 2024
cipherboy added a commit to cipherboy/openbao that referenced this issue Feb 22, 2024
This adds testing of API & SDK packages to the Makefile, letting
`make test` run them by default. This should fix issues where either is
broken as they weren't being run by most test executions.

Resolves: openbao#61

Signed-off-by: Alexander Scheel <alexander.m.scheel@gmail.com>
naphelps pushed a commit to cipherboy/openbao that referenced this issue Feb 27, 2024
This adds testing of API & SDK packages to the Makefile, letting
`make test` run them by default. This should fix issues where either is
broken as they weren't being run by most test executions.

Resolves: openbao#61

Signed-off-by: Alexander Scheel <alexander.m.scheel@gmail.com>
naphelps pushed a commit to cipherboy/openbao that referenced this issue Feb 27, 2024
This adds testing of API & SDK packages to the Makefile, letting
`make test` run them by default. This should fix issues where either is
broken as they weren't being run by most test executions.

Resolves: openbao#61

Signed-off-by: Alexander Scheel <alexander.m.scheel@gmail.com>
naphelps pushed a commit that referenced this issue Feb 27, 2024
This adds testing of API & SDK packages to the Makefile, letting
`make test` run them by default. This should fix issues where either is
broken as they weren't being run by most test executions.

Resolves: #61

Signed-off-by: Alexander Scheel <alexander.m.scheel@gmail.com>
Gabrielopesantos pushed a commit to Gabrielopesantos/openbao that referenced this issue Mar 1, 2024
This adds testing of API & SDK packages to the Makefile, letting
`make test` run them by default. This should fix issues where either is
broken as they weren't being run by most test executions.

Resolves: openbao#61

Signed-off-by: Alexander Scheel <alexander.m.scheel@gmail.com>
cipherboy added a commit to cipherboy/openbao that referenced this issue Mar 2, 2024
This retains the original HashiCorp upstream build & test pipelines,
cleaning them up for OpenBao and removing HashiCorp internal tooling
references that aren't necessary for us.

The CI pipeline currently fails with test errors and commenting will
need to be tested on the main repository with an appropriately scoped
token. However, builds pass and produce usable, unsigned artifacts.

This can form the basis of a proper (signed) release pipeline
eventually, taking actions from the build stage of the tagged
release commit and signing and verifying them.

In order to fix CI, some changes to the Go modules were done, removing
redundant tooling packages and re-adding the kubernetes integration
tests. This also fixes CI to correctly run api & sdk tests, fixing openbao#61
again.

Removed, unnecessary actions:

 - actionlint was used to allow-list actions upstream,
 - add-hashicorp-contributed-label was used to add a label to internal
   PRs for visibility,
 - backport was the tool to automatically backport PRs,
 - milestone-checker was used to ensure PRs had appropriate milestones
   prior to merge,
 - oss was used to classify issues against the specified label category
 - remove-labels was used to clean up issues & PRs
 - security-scan requires internal tooling not made public
 - test-ci-bootstrap & test-ci-cleanup are both part of the complex Enos
   integration tests, which were removed in
   85455fb due to resource
   requirements.

Resolves: openbao#31
Resolves: openbao#152
Related: openbao#153

Signed-off-by: Alexander Scheel <alexander.m.scheel@gmail.com>
cipherboy added a commit to cipherboy/openbao that referenced this issue Mar 3, 2024
This retains the original HashiCorp upstream build & test pipelines,
cleaning them up for OpenBao and removing HashiCorp internal tooling
references that aren't necessary for us.

The CI pipeline currently fails with test errors and commenting will
need to be tested on the main repository with an appropriately scoped
token. However, builds pass and produce usable, unsigned artifacts.

This can form the basis of a proper (signed) release pipeline
eventually, taking actions from the build stage of the tagged
release commit and signing and verifying them.

In order to fix CI, some changes to the Go modules were done, removing
redundant tooling packages and re-adding the kubernetes integration
tests. This also fixes CI to correctly run api & sdk tests, fixing openbao#61
again.

Removed, unnecessary actions:

 - actionlint was used to allow-list actions upstream,
 - add-hashicorp-contributed-label was used to add a label to internal
   PRs for visibility,
 - backport was the tool to automatically backport PRs,
 - milestone-checker was used to ensure PRs had appropriate milestones
   prior to merge,
 - oss was used to classify issues against the specified label category
 - remove-labels was used to clean up issues & PRs
 - security-scan requires internal tooling not made public
 - test-ci-bootstrap & test-ci-cleanup are both part of the complex Enos
   integration tests, which were removed in
   85455fb due to resource
   requirements.

Resolves: openbao#31
Resolves: openbao#152
Related: openbao#153

Signed-off-by: Alexander Scheel <alexander.m.scheel@gmail.com>
cipherboy added a commit to cipherboy/openbao that referenced this issue Mar 3, 2024
This retains the original HashiCorp upstream build & test pipelines,
cleaning them up for OpenBao and removing HashiCorp internal tooling
references that aren't necessary for us.

The CI pipeline currently fails with test errors and commenting will
need to be tested on the main repository with an appropriately scoped
token. However, builds pass and produce usable, unsigned artifacts.

This can form the basis of a proper (signed) release pipeline
eventually, taking actions from the build stage of the tagged
release commit and signing and verifying them.

In order to fix CI, some changes to the Go modules were done, removing
redundant tooling packages and re-adding the kubernetes integration
tests. This also fixes CI to correctly run api & sdk tests, fixing openbao#61
again.

Removed, unnecessary actions:

 - actionlint was used to allow-list actions upstream,
 - add-hashicorp-contributed-label was used to add a label to internal
   PRs for visibility,
 - backport was the tool to automatically backport PRs,
 - milestone-checker was used to ensure PRs had appropriate milestones
   prior to merge,
 - oss was used to classify issues against the specified label category
 - remove-labels was used to clean up issues & PRs
 - security-scan requires internal tooling not made public
 - test-ci-bootstrap & test-ci-cleanup are both part of the complex Enos
   integration tests, which were removed in
   85455fb due to resource
   requirements.

Resolves: openbao#31
Resolves: openbao#42
Resolves: openbao#152
Related: openbao#153

Signed-off-by: Alexander Scheel <alexander.m.scheel@gmail.com>
Gabrielopesantos pushed a commit to Gabrielopesantos/openbao that referenced this issue Mar 4, 2024
This adds testing of API & SDK packages to the Makefile, letting
`make test` run them by default. This should fix issues where either is
broken as they weren't being run by most test executions.

Resolves: openbao#61

Signed-off-by: Alexander Scheel <alexander.m.scheel@gmail.com>
naphelps pushed a commit to cipherboy/openbao that referenced this issue Mar 5, 2024
This retains the original HashiCorp upstream build & test pipelines,
cleaning them up for OpenBao and removing HashiCorp internal tooling
references that aren't necessary for us.

The CI pipeline currently fails with test errors and commenting will
need to be tested on the main repository with an appropriately scoped
token. However, builds pass and produce usable, unsigned artifacts.

This can form the basis of a proper (signed) release pipeline
eventually, taking actions from the build stage of the tagged
release commit and signing and verifying them.

In order to fix CI, some changes to the Go modules were done, removing
redundant tooling packages and re-adding the kubernetes integration
tests. This also fixes CI to correctly run api & sdk tests, fixing openbao#61
again.

Removed, unnecessary actions:

 - actionlint was used to allow-list actions upstream,
 - add-hashicorp-contributed-label was used to add a label to internal
   PRs for visibility,
 - backport was the tool to automatically backport PRs,
 - milestone-checker was used to ensure PRs had appropriate milestones
   prior to merge,
 - oss was used to classify issues against the specified label category
 - remove-labels was used to clean up issues & PRs
 - security-scan requires internal tooling not made public
 - test-ci-bootstrap & test-ci-cleanup are both part of the complex Enos
   integration tests, which were removed in
   85455fb due to resource
   requirements.

Resolves: openbao#31
Resolves: openbao#42
Resolves: openbao#152
Related: openbao#153

Signed-off-by: Alexander Scheel <alexander.m.scheel@gmail.com>
naphelps pushed a commit that referenced this issue Mar 5, 2024
This retains the original HashiCorp upstream build & test pipelines,
cleaning them up for OpenBao and removing HashiCorp internal tooling
references that aren't necessary for us.

The CI pipeline currently fails with test errors and commenting will
need to be tested on the main repository with an appropriately scoped
token. However, builds pass and produce usable, unsigned artifacts.

This can form the basis of a proper (signed) release pipeline
eventually, taking actions from the build stage of the tagged
release commit and signing and verifying them.

In order to fix CI, some changes to the Go modules were done, removing
redundant tooling packages and re-adding the kubernetes integration
tests. This also fixes CI to correctly run api & sdk tests, fixing #61
again.

Removed, unnecessary actions:

 - actionlint was used to allow-list actions upstream,
 - add-hashicorp-contributed-label was used to add a label to internal
   PRs for visibility,
 - backport was the tool to automatically backport PRs,
 - milestone-checker was used to ensure PRs had appropriate milestones
   prior to merge,
 - oss was used to classify issues against the specified label category
 - remove-labels was used to clean up issues & PRs
 - security-scan requires internal tooling not made public
 - test-ci-bootstrap & test-ci-cleanup are both part of the complex Enos
   integration tests, which were removed in
   85455fb due to resource
   requirements.

Resolves: #31
Resolves: #42
Resolves: #152
Related: #153

Signed-off-by: Alexander Scheel <alexander.m.scheel@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants