encoding/json: Decoding gives errPhase when unmarshaling a non-string literal into a TextUnmarshaler

[cespare]

updated

When decoding a non-string literal into an encoding.TextUnmarshaler (such as net.IP), json.Unmarshal returns errPhase ("JSON decoder out of sync - data changing underfoot?").

Reading the code, it seems like it's supposed to return a type error when this happens
(source). But the errPhase ends up taking precedence and obscuring this message.

I'm not sure why UnmarshalText should only work for string literals, though.

Example play here.

Code:

package main

import (
  "encoding/json"
  "fmt"
  "net"
  "strconv"
)

type X int

func (x *X) UnmarshalText(b []byte) error {
  n, err := strconv.ParseInt(string(b), 10, 64)
  if err != nil {
    return err
  }
  *x = X(n)
  return nil
}

func main() {
  var x X
  err := x.UnmarshalText([]byte("123"))
  fmt.Println(x, err) // 123 <nil>

  // Works if the string is quoted
  err = json.Unmarshal([]byte(`"234"`), &x)
  fmt.Println(x, err) // 234 <nil>

  // Otherwise it gives an errPhase
  err = json.Unmarshal([]byte("123"), &x)
  fmt.Println(err) // JSON decoder out of sync - data changing underfoot?
  err = json.Unmarshal([]byte("true"), &x)
  fmt.Println(err) // JSON decoder out of sync - data changing underfoot?
  err = json.Unmarshal([]byte("null"), &x)
  fmt.Println(err) // JSON decoder out of sync - data changing underfoot?

  // Happens with real types like net.IP

  var ip net.IP
  err = json.Unmarshal([]byte("8080"), &ip)
  fmt.Println(err) // JSON decoder out of sync - data changing underfoot?

}

[awly]

Note that this doesn't happen when json value is quoted as expected by TextUnmarshaler.

package main

import (
    "encoding/json"
    "errors"
    "fmt"
)

type X struct{}

func (x *X) UnmarshalText(b []byte) error {
    return errors.New("Hi!")
}

func main() {
    var x X
    err := json.Unmarshal([]byte("\"123\""), &x)
    fmt.Println(err) // Hi!
}

[cespare]

@alytvynov This bug report is for non-string literals. It is true that it passes the correct error for string literals.

I just noticed that this happens even when the TextUnmarshaler doesn't return an error. Updating the description now.

[awly]

@cespare sure, just pointing that out as a possible hint for debugging.

[dominikh]

@rsc: Two questions about your fix:

1. Can we infer from your fix that unmarshaling a non-string value, even null, into a TextUnmarshaler is always an error? That is, null can't be unmarshaled into a *net.IP, even though both *net.IP and net.IP can be nil?

2. The error now reads cannot unmarshal string into Go value of type [...] when trying to unmarshal null. Should null really be considered a string, or should an issue be filed for this?