Marathon Service Discovery in Prometheus picks up only the first port of a task as target

[rohitChaware]

What did you do?
I am running Prometheus configured with Marathon service discovery in a docker container. I have an app running in Marathon which exposes more than 2 ports and I have /metrics available on a port(say X) exposed by the app which is not the first port listed in the app definition JSON file.

What did you expect to see?
Expected Prometheus to discover the app from Marathon, show the app with port X as an endpoint in targets and also show the endpoint's state as UP.

What did you see instead? Under which circumstances?
Prometheus discovers the app, shows the app with port which is listed first in the app definition JSON file(not X) as an endpoint and shows the state of the endpoint as DOWN.

Environment

• System information:
  Linux 4.4.0-36-generic x86_64

• Prometheus version: Output of <prometheus -version> from the Prometheus container
  prometheus, version 1.5.0 (branch: master, revision: d840f2c)
  build user: root@a04ed5b536e3
  build date: 20170123-13:56:24
  go version: go1.7.4

• Prometheus configuration file:

global:
  scrape_interval: 5s

scrape_configs:
  - job_name: 'job_x'
    marathon_sd_configs:
    - servers:
      - 'http://100.14.6.158:8005'

Possible resolution:
The function targetForTask() in the file 'discovery/marathon/marathon.go' returns only the first port of a task as target.

func targetForTask(task *Task) string {
	return net.JoinHostPort(task.Host, fmt.Sprintf("%d", task.Ports[0]))
}

The function targetsForApp in the file 'discovery/marathon/marathon.go' can be modified, as below, to consider all the ports of a task as targets:

func targetsForApp(app *App) []model.LabelSet {
	targets := make([]model.LabelSet, 0, len(app.Tasks))
	for _, t := range app.Tasks {
		if len(t.Ports) == 0 {
			continue
		}
		for _, port := range t.Ports {
			target := targetForTask(&t, port)
			targets = append(targets, model.LabelSet{
				model.AddressLabel: model.LabelValue(target),
				taskLabel:          model.LabelValue(t.ID),
			})
		}
	}
	return targets
}

func targetForTask(task *Task, port uint32) string {
	return net.JoinHostPort(task.Host, fmt.Sprintf("%d", port))
 }

(Verified)With just the above modification, Prometheus shows all the ports of a task as endpoints under targets.
Attaching the file 'discovery/marathon/marathon.go' as a marathon.txt file(.go file not supported as attachment).

[grobie]

Fixed with #2506.

[lock]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.