Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Uses offset instead of line number in JSONSyntaxError #97990

Merged
merged 1 commit into from Jan 15, 2021
+11 −31
Merged

Uses offset instead of line number in JSONSyntaxError #97990

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
1 change: 0 additions & 1 deletion staging/src/k8s.io/apimachinery/pkg/util/yaml/BUILD
View file
Open in desktop
Expand Up @@ -19,7 +19,6 @@ go_library(
importpath = "k8s.io/apimachinery/pkg/util/yaml",
deps = [
"//staging/src/k8s.io/apimachinery/pkg/util/json:go_default_library",
"//vendor/k8s.io/klog/v2:go_default_library",
"//vendor/sigs.k8s.io/yaml:go_default_library",
],
)
Expand Down
36 changes: 8 additions & 28 deletions staging/src/k8s.io/apimachinery/pkg/util/yaml/decoder.go
View file
Open in desktop
Expand Up @@ -22,13 +22,11 @@ import (
"encoding/json"
"fmt"
"io"
"io/ioutil"
"strings"
"unicode"

jsonutil "k8s.io/apimachinery/pkg/util/json"

"k8s.io/klog/v2"
"sigs.k8s.io/yaml"
)

Expand Down Expand Up @@ -215,16 +213,15 @@ type YAMLOrJSONDecoder struct {
bufferSize int

decoder decoder
rawData []byte
}

type JSONSyntaxError struct {
Line int
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you leave Line here with a note that it isn't populated currently?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done (though I think it will only create confusion for the one who would like to use it).

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah I can see going either way, I was mostly thinking in terms of not breaking anyone using this public type at compile time. I can see people thinking that'd actually be a good thing. In this case I lean towards keeping it, but I don't feel super strongly.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I removed it

Err error
Offset int64
Err error
}

func (e JSONSyntaxError) Error() string {
return fmt.Sprintf("json: line %d: %s", e.Line, e.Err.Error())
return fmt.Sprintf("json: offset %d: %s", e.Offset, e.Err.Error())
}

type YAMLSyntaxError struct {
Expand All @@ -250,35 +247,18 @@ func NewYAMLOrJSONDecoder(r io.Reader, bufferSize int) *YAMLOrJSONDecoder {
// provide object, or returns an error.
func (d *YAMLOrJSONDecoder) Decode(into interface{}) error {
if d.decoder == nil {
buffer, origData, isJSON := GuessJSONStream(d.r, d.bufferSize)
buffer, _, isJSON := GuessJSONStream(d.r, d.bufferSize)
if isJSON {
d.decoder = json.NewDecoder(buffer)
d.rawData = origData
} else {
d.decoder = NewYAMLToJSONDecoder(buffer)
}
}
err := d.decoder.Decode(into)
if jsonDecoder, ok := d.decoder.(*json.Decoder); ok {
if syntax, ok := err.(*json.SyntaxError); ok {
data, readErr := ioutil.ReadAll(jsonDecoder.Buffered())
if readErr != nil {
klog.V(4).Infof("reading stream failed: %v", readErr)
}
js := string(data)

// if contents from io.Reader are not complete,
// use the original raw data to prevent panic
if int64(len(js)) <= syntax.Offset {
js = string(d.rawData)
}

start := strings.LastIndex(js[:syntax.Offset], "\n") + 1
line := strings.Count(js[:start], "\n")
return JSONSyntaxError{
Line: line,
Err: fmt.Errorf(syntax.Error()),
}
if syntax, ok := err.(*json.SyntaxError); ok {
return JSONSyntaxError{
Offset: syntax.Offset,
Err: syntax,
}
}
return err
Expand Down
5 changes: 3 additions & 2 deletions staging/src/k8s.io/apimachinery/pkg/util/yaml/decoder_test.go
View file
Open in desktop
Expand Up @@ -242,8 +242,9 @@ func TestDecodeBrokenJSON(t *testing.T) {
if err == nil {
t.Fatal("expected error with json: prefix, got no error")
}
if !strings.HasPrefix(err.Error(), "json: line 3:") {
t.Fatalf("expected %q to have 'json: line 3:' prefix", err.Error())
const msg = `json: offset 28: invalid character '"' after object key:value pair`
if msg != err.Error() {
t.Fatalf("expected %q, got %q", msg, err.Error())
}
}

Expand Down