Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

googleapi: Error 400: Parse Error, parseError #389

Closed
derekchiang opened this issue Oct 3, 2016 · 1 comment

Comments

Projects
None yet
2 participants
@derekchiang
Copy link

commented Oct 3, 2016

I'm getting this error when putting a couple hundreds of objects concurrently:

googleapi: Error 400: Parse Error, parseError

This is the code that you can use to reproduce this issue:

package main

import (
    "context"
    "fmt"
    "math/rand"
    "sync"

    "cloud.google.com/go/storage"
    "golang.org/x/sync/errgroup"
)

const letterBytes = "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ"

func RandStringBytes(n int) string {
    b := make([]byte, n)
    for i := range b {
        b[i] = letterBytes[rand.Intn(len(letterBytes))]
    }
    return string(b)
}

func main() {
    ctx := context.Background()
    client, err := storage.NewClient(ctx)
    if err != nil {
        panic(err)
    }

    bkt := client.Bucket("some bucket")

    for j := 0; j < 20; j++ {
        j := j
        var g errgroup.Group
        for i := 0; i < (j+1)*50; i++ {
            i := i
            g.Go(func() error {
                objName := fmt.Sprintf("%d-%d", j, i)
                obj := bkt.Object(objName)
                w := obj.NewWriter(ctx)
                bytes := RandStringBytes(1024)
                _, err := fmt.Fprint(w, bytes)
                if err != nil {
                    return err
                }
                return w.Close()
            })
        }
        if err := g.Wait(); err != nil {
            panic(err)
        }
    }
}

The code simply ramps up the request rate until it hits the error. On my machine, it seems like I consistently get the error when putting around 200 objects concurrently.

@jba jba self-assigned this Oct 4, 2016

@jba

This comment has been minimized.

Copy link
Contributor

commented Oct 24, 2016

I think this was due to golang/go#17071, which is fixed in go 1.7.3. Not only does the bug seem to match up with these symptoms, but also I was able to replicate this issue up until last Thursday or so, which seems to match up pretty well with when 1.7.3 was pushed to my machine.

So update your Go version and see if that fixes it. I'm going to assume it will.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.