Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ExecutionApp should print out custom stats like Job #1764

Open
benpence opened this issue Jan 8, 2018 · 2 comments
Open

ExecutionApp should print out custom stats like Job #1764

benpence opened this issue Jan 8, 2018 · 2 comments

Comments

@benpence
Copy link
Contributor

benpence commented Jan 8, 2018

Job prints out custom stats in #run(...); ExecutionApp does not. It might be useful to get the stats across an entire Execution run, so I think we should print these out (and expose this API point a little better).

I'm assuming the reason we didn't do this is an oversight. Or is there some design decision?

@johnynek
Copy link
Collaborator

johnynek commented Jan 8, 2018

you can access the stats in Execution so you can print these out yourself.

Execution.getAndResetCounters or getCounters + flatMap and println should be all you need.

You can do this in a function in your own repo:

def withCustomCounters[T](ex: Execution[T]): Execution[T] =
  for {
    t <- ex
    counters <- Execution.getCounters
    _ = printCustom(counters)
  } yield t

Something like that.

@benpence
Copy link
Contributor Author

benpence commented Jan 8, 2018

Ya I saw that when looking through it. Something along the lines of

job
  .getCounters
  .map {
    case (_, counters) =>
      println("Dumping custom counters:")

      counters.keys.foreach { key =>
        val group = key.group
        val counter = key.counter
        val value = counters(key)

        println(s"$counter\t$value")
      }
  }
  .waitFor(conf, mode).get

in ExecutionApp.main

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants