Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding a better error message for download mode #1492

Open
wants to merge 25 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 5 commits
Commits
Show all changes
25 commits
Select commit Hold shift + click to select a range
0224e53
Adding a better error message for download mode
arschles Dec 12, 2019
fa2631c
Fixing test
arschles Dec 12, 2019
834eba4
Making some progress toward removing the specific config error
arschles Dec 13, 2019
f45a03b
Fixing last test compile error
arschles Dec 13, 2019
4715103
Merge branch 'master' into config-msgs
arschles Dec 19, 2019
e928a0f
Merge branch 'master' into config-msgs
arschles Feb 14, 2020
32e7c28
Moving downloadModeURL to package-global
arschles Feb 14, 2020
a846557
using a valid HTTP error code instead of -1
arschles Feb 14, 2020
25b5860
removing the config.Validate function
arschles Feb 14, 2020
36b6d62
making the DownloadMode field required
arschles Feb 14, 2020
2c6b90b
validating that the port starts with ':'
arschles Feb 14, 2020
5b85142
Merge branch 'master' into config-msgs
arschles Feb 21, 2020
65662f5
Not passing an op to Validate functions
arschles Feb 21, 2020
e962a4d
Not checking for error strings in mode test
arschles Feb 21, 2020
1b29dcf
more doc for Validator
arschles Feb 21, 2020
d6a7cbd
removing struct tag for config port validation
arschles Feb 21, 2020
de71cb7
Merge branch 'master' into config-msgs
arschles Mar 10, 2020
cc70e12
Removing the unnecessary validator interface
arschles Mar 10, 2020
a5eda16
Joining error strings with newline+tabs instead of just a space
arschles Mar 10, 2020
198d574
s/retFile/df
arschles Mar 10, 2020
ee8f4d1
validating each download path in DownloadFile.Validate
arschles Mar 10, 2020
2cc9f84
Checking for custom or file modes
arschles Mar 10, 2020
9695c36
Checking top-level DownloadURL
arschles Mar 10, 2020
726b77c
Merge branch 'config-msgs' of github.com:arschles/athens into config-…
arschles Mar 10, 2020
4320101
validating the download path mode only in DownloadPath.Validate
arschles Mar 10, 2020
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
3 changes: 1 addition & 2 deletions cmd/proxy/actions/app.go
Expand Up @@ -132,8 +132,7 @@ func App(conf *config.Config) (http.Handler, error) {
lggr,
conf,
); err != nil {
err = fmt.Errorf("error adding proxy routes (%s)", err)
return nil, err
return nil, fmt.Errorf("Error initializing Athens:\n%s", err)
}

h := &ochttp.Handler{
Expand Down
4 changes: 4 additions & 0 deletions cmd/proxy/main.go
Expand Up @@ -15,6 +15,7 @@ import (
"github.com/gomods/athens/cmd/proxy/actions"
"github.com/gomods/athens/pkg/build"
"github.com/gomods/athens/pkg/config"
"github.com/gomods/athens/pkg/errors"
)

var (
Expand All @@ -32,6 +33,9 @@ func main() {
if err != nil {
log.Fatalf("could not load config file: %v", err)
}
if err := conf.Validate(errors.Op("main")); err != nil {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We already have a validateConfig function here: https://github.com/gomods/athens/blob/master/pkg/config/config.go#L238

It gets automatically called when you call config.Load

So this is a bit confusing :)

In the same fashion, the mode.Validate function in this case is getting called twice, once here and once in mode.NewFile.

I think we can just:

  1. remove this line here
  2. remove config.Validate function
  3. make mode.Validate private

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@marwan-at-work I added a Validate function on config structs so that we can nest validations. I'll try to move all of the logic into validate struct tags though

log.Fatal(err)
}

handler, err := actions.App(conf)
if err != nil {
Expand Down
7 changes: 7 additions & 0 deletions pkg/config/config.go
Expand Up @@ -58,6 +58,13 @@ type Config struct {
Storage *StorageConfig
}

// Validate checks various values in c that are commonly improperly set.
// This function is not exhaustive, so c is not guaranteed to be valid even if
// it doesn't return nil
func (c Config) Validate(op errors.Op) error {
return c.DownloadMode.Validate(op)
}

// EnvList is a list of key-value environment
// variables that are passed to the Go command
type EnvList []string
Expand Down
69 changes: 51 additions & 18 deletions pkg/download/mode/mode.go
Expand Up @@ -18,15 +18,29 @@ type Mode string

// DownloadMode constants. For more information see config.dev.toml
const (
Sync Mode = "sync"
Async Mode = "async"
Redirect Mode = "redirect"
AsyncRedirect Mode = "async_redirect"
None Mode = "none"
downloadModeErr = "download mode is not set"
invalidModeErr = "unrecognized download mode: %s"
Sync Mode = "sync"
Async Mode = "async"
Redirect Mode = "redirect"
AsyncRedirect Mode = "async_redirect"
None Mode = "none"
)

// Validate ensures that m is a valid mode. If this function returns nil, you are
// guaranteed that m is valid
func (m Mode) Validate(op errors.Op) error {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of it taking an op, just make a mode.Validate operation under this line and wrap the error with it. This way, people can get the entire call stack by the time the server fails to boot up.

switch m {
case Sync, Async, Redirect, AsyncRedirect, None:
return nil
default:
return errors.Config(
op,
"mode",
fmt.Sprintf("%s isn't a valid value.", m),
"https://docs.gomods.io/configuration/download/",
)
}
}

// DownloadFile represents a custom HCL format of
// how to handle module@version requests that are
// not found in storage.
Expand All @@ -44,17 +58,30 @@ type DownloadPath struct {
DownloadURL string `hcl:"downloadURL,optional"`
}

func (d DownloadPath) validate(op errors.Op) error {
if d.DownloadURL == "" && (d.Mode == Redirect || d.Mode == AsyncRedirect) {
return errors.Config(
op,
fmt.Sprintf("DownloadURL (inside %s in the download file)", d.Pattern),
"You must set a value when the download mode is 'redirect' or 'async_redirect'",
"https://docs.gomods.io/configuration/download/",
)
}
return nil
}

// NewFile takes a mode and returns a DownloadFile.
// Mode can be one of the constants declared above
// or a custom HCL file. To pass a custom HCL file,
// you can either point to a file path by passing
// file:/path/to/file OR custom:<base64-encoded-hcl>
// directly.
func NewFile(m Mode, downloadURL string) (*DownloadFile, error) {
const downloadModeURL = "https://docs.gomods.io/configuration/download/"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would make this a package level constant (private) as it is being used 3 more times in the pr

const op errors.Op = "downloadMode.NewFile"

if m == "" {
return nil, errors.E(op, downloadModeErr)
if err := m.Validate(op); err != nil {
return nil, err
}

if strings.HasPrefix(string(m), "file:") {
Expand All @@ -72,12 +99,11 @@ func NewFile(m Mode, downloadURL string) (*DownloadFile, error) {
return parseFile(bts)
}

switch m {
case Sync, Async, Redirect, AsyncRedirect, None:
return &DownloadFile{Mode: m, DownloadURL: downloadURL}, nil
default:
return nil, errors.E(op, errors.KindBadRequest, fmt.Sprintf(invalidModeErr, m))
retFile := &DownloadFile{Mode: m, DownloadURL: downloadURL}
marwan-at-work marked this conversation as resolved.
Show resolved Hide resolved
if err := retFile.validate(op); err != nil {
return nil, err
}
return retFile, nil
}

// parseFile parses an HCL file according to the
Expand All @@ -93,19 +119,26 @@ func parseFile(file []byte) (*DownloadFile, error) {
if dig.HasErrors() {
return nil, errors.E(op, dig.Error())
}
if err := df.validate(); err != nil {
if err := df.validate(op); err != nil {
return nil, errors.E(op, err)
}
return &df, nil
}

func (d *DownloadFile) validate() error {
const op errors.Op = "downloadMode.validate"
func (d *DownloadFile) validate(op errors.Op) error {
for _, p := range d.Paths {
if err := p.Mode.Validate(op); err != nil {
return err
}
switch p.Mode {
case Sync, Async, Redirect, AsyncRedirect, None:
default:
return errors.E(op, fmt.Errorf("unrecognized mode for %v: %v", p.Pattern, p.Mode))
return errors.Config(
op,
fmt.Sprintf("mode (in pattern %v", p.Pattern),
fmt.Sprintf("%s is unrecognized", p.Mode),
"https://docs.gomods.io/configuration/download/",
)
}
}
return nil
Expand Down
8 changes: 5 additions & 3 deletions pkg/download/mode/mode_test.go
@@ -1,8 +1,9 @@
package mode

import (
"fmt"
"testing"

"github.com/gomods/athens/pkg/errors"
)

var testCases = []struct {
Expand Down Expand Up @@ -119,6 +120,7 @@ func TestMode(t *testing.T) {
}

func TestNewFile_err(t *testing.T) {
const op = errors.Op("downloadMode.NewFile")
tc := []struct {
name string
mode Mode
Expand All @@ -127,12 +129,12 @@ func TestNewFile_err(t *testing.T) {
{
name: "empty mode",
mode: "",
expected: downloadModeErr,
expected: Mode("").Validate(op).Error(),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hmm, we probably shouldn't do string matching here: either

  1. change expected string to hasErr bool
  2. or, change expected string to expected errors.Kind

},
{
name: "invalid mode",
mode: "invalidMode",
expected: fmt.Sprintf(invalidModeErr, "invalidMode"),
expected: Mode("invalidMode").Validate(op).Error(),
},
}
for _, c := range tc {
Expand Down
22 changes: 22 additions & 0 deletions pkg/errors/config.go
@@ -0,0 +1,22 @@
package errors

import (
"fmt"
"strings"
)

// Config returns an error specifically tailored for reporting errors with configuration
// values. You can check for these errors by calling errors.Is(err, KindConfigError)
// (from the github.com/gomods/athens/pkg/errors package).
//
// Generally these kinds of errors should make Athens crash because it was configured
// improperly
func Config(op Op, field, helpText, url string) error {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

will this ever be called from outside the config package? why not just define it there and define it locally to minimize the API surface of this package?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nope. I put it here just because I think it makes more sense to group a configuration error under the errors package. I'm willing to be convinced otherwise 😄

slc := []string{
fmt.Sprintf("There was a configuration error with %s. %s", field, helpText),
}
if url != "" {
slc = append(slc, fmt.Sprintf("Please see %s for more information.", url))
}
return E(op, KindConfigError, strings.Join(slc, " "))
marwan-at-work marked this conversation as resolved.
Show resolved Hide resolved
}
1 change: 1 addition & 0 deletions pkg/errors/errors.go
Expand Up @@ -18,6 +18,7 @@ const (
KindRateLimit = http.StatusTooManyRequests
KindNotImplemented = http.StatusNotImplemented
KindRedirect = http.StatusMovedPermanently
KindConfigError = -1
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe the HTTP protocol gives you some room for custom codes
In this case this is a client (user) error so maybe we can give it a code between 451 and 499: https://golang.org/pkg/net/http/#pkg-constants

It doesn't really matter but just being a bit more conventional :)

Copy link

@bluekeyes bluekeyes Dec 20, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

422 / http.StatusUnprocessableEntity is a common response when a client provides syntactically valid but semantically invalid objects to an API and could be appropriate here, if you want to stick with valid HTTP codes.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@marwan-at-work fair enough! I'll take @bluekeyes's suggestion and go with 422, just so we can use a constant.

)

// Error is an Athens system error.
Expand Down