Error while unmarshaling network messages

[Comradin]

Hello,

I am trying to analyze the resource usage of a website with the help of the chromedp library. When enabling the network domain I get corrupt json messages.

From my go.mod file the versions I am using are:
github.com/chromedp/cdproto v0.0.0-20190429085128-1aa4f57ff2a9
github.com/chromedp/chromedp v0.3.1

A short example of my code:

func main() {
  cpctx, cancel := chromedp.NewContext(ctx,
  		     chromedp.WithDebugf(devToolHandler))
  defer cancel()

  err := chromedp.Run(cpctx,
           network.Enable(),
           chromedp.Navigate(baseUrl.String()),
             ...
         )
}

// The handler being used in the context
func devToolHandler(s string, is ...interface{}) {
	/*
	   Uncomment the following line to have a log of the events
	   log.Printf(s, is...)
	*/
	/*
	   We need this to be on a separate gorutine
	   otherwise we block the browser and we don't receive messages
	*/
	go func() {
		for _, elem := range is {
			var msg cdproto.Message
			// The CDP messages are sent as strings so we need to convert them back
			err := json.Unmarshal([]byte(fmt.Sprintf("%s", elem)), &msg)
			// possible source of empty msg!!!!!!!!!!!!!
			if err != nil {
				log.Println(err)
				log.Printf("Faulty element:\n%v\n", fmt.Sprintf("%s", elem))
			}

			msgChan <- msg
		}
	}()
}

Some examples of those corrupt messages
The message has been cut off inside the targetid field:

2019/08/16 11:27:14 unexpected end of JSON input
2019/08/16 11:27:14 Faulty element:
{"method":"Target.receivedMessageFromTarget","params":{"sessionId":"12C28C69BFC88E78A8BD2AA83AB2D475","message":"{\"method\":\"Network.loadingFinished\",\"params\":{\"requestId\":\"1000030524.136\",\"timestamp\":180150.31724,\"encodedDataLength\":31389,\"shouldReportCorbBlocking\":false}}","targetId":"6296FD1F8C4D089

An example message from the network domain, this time the error occures after the targetId:

2019/08/16 11:27:14 invalid character 'a' after top-level value
2019/08/16 11:27:14 Fautly element:
{"method":"Target.receivedMessageFromTarget","params":{"sessionId":"12C28C69BFC88E78A8BD2AA83AB2D475","message":"{\"method\":\"Network.loadingFinished\",\"params\":{\"requestId\":\"1000030524.95\",\"timestamp\":180150.112695,\"encodedDataLength\":23093,\"shouldReportCorbBlocking\":false}}","targetId":"6296FD1F8C4D08988ADD6EC222A30C3F"}}ary.com/flyby/t_thumbOWP/15584289868777.png\",\"status\":200,\"statusText\":\"\",\"headers\":{\"date\":\"Fri, 16 Aug 2019 09:27:14 GMT\",\"via\":\"1.1 varnish\",\"age\":\"1301358\",<CUTOFF>}}

It is like the beginning of the URL is missing in all these messages. Note, that the end of the message is complete, I have cut off the rest of the line to shorten the message a bit.

First I did some experiments with the EnableParams (WithMaxTotalBufferSizes) until I realized that there are filters by message size so that overly large messages won't hit my code.

I guess either I have made some mistake with the setup of my chromedp.NewContext() or there is a bug when the slice of interfaces for is is being created. I tracked the call chain into the chromedp conn.go file Read() func.

To me the observed behaviour could be explained by two buffers are copied into a third and the first overwrites parts of the second?

Kind regards,
Marcus

[Comradin]

Coming back after the weekend Im quite sure that this issue should not be in the cdproto package but in the chromedp package. Will close this one and move to the main package.