How to accses Outputs from custom callback with generator

[leanderloew]

Greetings,
I was following the example of the custom callback on the rstudio website and there you could access the stored list by history$losses. However with the same callback this is not possible when replacing fit by fit generator.
Thanks for your help

[jjallaire]

That might be just what is provided by the Keras generator interface.

Could you make sure that your callback has a reference to the model and access the history directly off of the model object?

[leanderloew]

Hi, so I managed to store the results this way:

AUC_Test <- AUC_fun(Pospred = test_pos_neg[[1]],Negpred = test_pos_neg[[2]])
         
self$model$history$history$AUC_Test <- c(self$model$history$history$AUC_Test, AUC_Test)

Is this how it is supposed to be/ what you mean? Seems a bit strange to me. Besides how can I Print out my metrics in a simillar way to the metrics included in the model?
simply using

print(AUC_Test)

doesnt seem to work.

Thanks for your help!

[jjallaire]

Note that this is exactly where custom metrics would be stored. Would writing a custom metric be a more elegant solution? (https://keras.rstudio.com/reference/metric_binary_accuracy.html#custom-metrics)

[leanderloew]

Custom metrics are calculated per batch right? If so that isnt really an option. Any idea how to print?