how to save the train and test accuracies to disk

[hzhz2020]

Dear Authors

I'm wondering is there an easy way to save the train accuracies (a list of accuracies) and test accuracies (a list of accuracies) on disk, maybe as a pkl file or npy file using your codebase? I saw that the train accuracy and test accuracy of each checkpoint are calculated with the eval_stats function, and the eval_stats function is called by the add_summaries function. I know we can use tensorboard to monitor the training progress, but I'm not sure how to save those accuracies on disk (I am also not sure which function called add_summaries and when)?

Sorry, I'm relatively new to tensorflow. It would really appreciate if you can point me to the right direction.

Many thanks!

[carlini]

The code is currently set up to log all of the data to tensorboard. The easiest way would probably be to just export the tensorboard data and process it directly.

If you must do it by modifying the code, then there are two ways you could do this:

1. The function eval_stats is called each epoch, and prints to stdout the train and test accuracies. You can access that directly here
   

   mixmatch/libml/train.py

   Lines 245 to 246 in 1011a1d

   	self.train_print('kimg %-5d accuracy train/valid/test %.2f %.2f %.2f' %
   	tuple([self.tmp.step >> 10] + accuracies))

2. If you need the mini-batch by mini-batch train losses, you'll need to make the model call return the loss values
   

   mixmatch/mixmatch.py

   Lines 104 to 107 in 1011a1d

   	return EasyDict(
   	x=x_in, y=y_in, label=l_in, train_op=train_op, tune_op=train_bn,
   	classify_raw=tf.nn.softmax(classifier(x_in, training=False)), # No EMA, for debugging.
   	classify_op=tf.nn.softmax(classifier(x_in, getter=ema_getter, training=False)))

   
   and then will need to update train_step
   

   mixmatch/libml/train.py

   Line 135 in 1011a1d

   self.tmp.step = train_session.run([self.ops.train_op, self.ops.update_step],

   
   so that it requests the new loss you add and save it to disk from there.

[hzhz2020]

Thanks a lot for the helpful comments!