make perfomrance pr-bot to use json output

[medyagh]

for more information ask @priyawadhwa

[Aut0R3V]

@priyawadhwa can you please explain about this issue

[priyawadhwa]

Hey @Aut0R3V -- we have a binary called mkcmp in our repo -- code can be found here:

1. https://github.com/kubernetes/minikube/blob/master/cmd/performance/mkcmp/cmd/mkcmp.go
2. https://github.com/kubernetes/minikube/tree/master/pkg/minikube/perf

mkcmp is responsible for comparing start time between two minikube binaries. It runs each binary three times and outputs information around how long it took. This output is pasted on our PRs by our PR bot (example)

You can build the binary locally by running:

make out/mkcmp

Right now, we split up how long minikube start takes by log, the code is in this file

Instead of timing by logs to STDOUT, it would be better if we used the JSON output feature in minikube to more accurately parse logs and match up how long different steps take.

This would require running minikube start in mkcmp with the --output=json flag and parsing the output correctly.

[Aut0R3V]

Thanks a lot for the detailed explanation!

[Aut0R3V]

/assign

[priyawadhwa]

@Aut0R3V thanks for taking this issue on!

[Aut0R3V]

@priyawadhwa I had some questions :

1] What changes are to be added at the already existing mkcmp workflow ?
[https://github.com/kubernetes/minikube/blob/master/cmd/performance/mkcmp/cmd/mkcmp.go]

2] Is this the correct defaultTimeFormat for the output :
type Status struct {
defaultStatusFormat = `host: {{.Host}}

kubelet: {{.Kubelet}}

apiserver: {{.APIServer}}

kubeconfig: {{.Kubeconfig}}
`
)
If not then
What would be the defaultTimeFormat for the output ?

[priyawadhwa]

Hey @Aut0R3V --

1. The code that needs to be changed is in this function:
   

   minikube/pkg/minikube/perf/logs.go

   Line 33 in fdab252

   func timeCommandLogs(cmd *exec.Cmd) (*result, error) {

Basically, instead of streaming logs from STDOUT and timing how long each of them takes, we want to swtich to streaming JSON logs. JSON log steps have the following format:

{"data":{"currentstep":"0","message":"minikube v1.12.1 on Darwin 10.14.6\n","name":"Initial Minikube Setup","totalsteps":"10"},"datacontenttype":"application/json","id":"68ff70ae-202b-4b13-8351-e9f060e8c56e","source":"https://minikube.sigs.k8s.io/","specversion":"1.0","type":"io.k8s.sigs.minikube.step"}

We want to parse out the "name" of the step from the JSON output and use that, instead of the full text output. That would mean replacing this line:

minikube/pkg/minikube/perf/logs.go

Line 54 in fdab252

log := scanner.Text()

with

log = <name from parsed JSON output>

2. I'm not sure what your question is, could you clarify?

This issue is definitely complicated, so thanks for your interest!

[Aut0R3V]

Thanks for your help!

[Aut0R3V]

@priyawadhwa is there any specific file where the logs would be coming to?

[priyawadhwa]

Hey @Aut0R3V we get the logs by reading STDOUT from this command as it is running:

minikube/pkg/minikube/perf/logs.go

Line 45 in fdab252

if err := cmd.Start(); err != nil {

[prezha]

@Aut0R3V, we haven't heard back from you for some time, are you still working on this issue?

[medyagh]

not needed anmore