How to change the logging setting for SGD and layerwise trainer?

[AminSuzani]

Hi,

I was wondering if there is a way to control the logging info for layerwise and SGD optimization? For example, I like to see the training error only every other 50 updates (not on each update). My training takes a couple days. Whenever I get to my computer I see the logging for the last hour at most, and I won't get a feeling of what's going on.

Thanks for your great package,
Amin

[kastnerkyle]

If you are using Linux (maybe Mac too?), try

python -u myfile.py 2>&1 | tee log.log

This should log all data to a file called log.log, while still letting you
see the last hour or so in the terminal. The -u is to run python in
unbuffered mode, which is what allows tee to work properly. While logging
thresholds may be nice, being able to see all execution is still
important!

On Fri, Jun 27, 2014 at 3:56 PM, Amin Suzani notifications@github.com
wrote:

Hi,

I was wondering if there is a way to control the logging info for
layerwise and SGD optimization? For example, I like to see the training
error only every other 50 updates (not on each update). My training takes a
couple days. Whenever I get to my computer I see the logging for the last
hour at most, and I won't get a feeling of what's going on.

Thanks for your great package,
Amin

—
Reply to this email directly or view it on GitHub
#27.

[lmjohns3]

I agree with @kastnerkyle that it's useful to have a log with lots of information in it, and the tee strategy is one that I use personally (then later you can grep through the on-disk log file and make quick learning curve graphs, etc.).

However, you can also configure the number of training batches that get processed per log line, by using the --train-batches (command-line) or train_batches (programmatic) arguments when running your model. Set this to a large number to see less-frequent updates.

[AminSuzani]

Thanks Kyle and Leif for your answers. using tee was a really good idea. Increasing train batches made the training faster, but led to less accuracy.