reports.org

Training information

Base model

Without pruning.

Epoch	Time (s)	Train Loss	Test Loss	Train Acc	Test Acc	Sparsity
1	108.65	35.958	27.779	0.68	0.78	0.0
2	111.31	24.467	24.578	0.84	0.84	0.0
3	98.24	19.873	20.623	0.87	0.87	0.0
4	109.19	17.795	22.941	0.89	0.87	0.0
5	106.14	16.155	20.806	0.9	0.87	0.0
6	105.55	14.864	21.017	0.91	0.87	0.0
7	103.66	13.645	22.307	0.92	0.87	0.0
8	104.53	11.989	21.826	0.93	0.86	0.0

Config

Basic pruning config.

Epoch	Time (s)	Train Loss	Test Loss	Train Acc	Test Acc	Sparsity
1	110.37	35.547	25.801	0.69	0.83	0.0
2	100.53	24.139	23.167	0.84	0.81	0.08
3	103.67	19.789	22.054	0.87	0.84	0.26
4	101.6	16.809	20.582	0.89	0.87	0.45
5	103.79	15.053	20.901	0.91	0.87	0.64
6	107.55	14.253	21.799	0.91	0.87	0.81
7	112.47	14.276	24.024	0.92	0.87	0.94
8	104.8	11.653	22.526	0.93	0.84	0.94
9	100.44	11.072	24.969	0.93	0.86	0.94
10	113.35	11.093	22.774	0.93	0.85	0.94

Config

Pruning with higher frequency. Also increased q.

Epoch	Time (s)	Train Loss	Test Loss	Train Acc	Test Acc	Sparsity
1	96.19	34.995	23.946	0.7	0.85	0.0
2	91.82	23.645	20.618	0.85	0.87	0.12
3	97.14	18.799	21.472	0.88	0.87	0.35
4	87.23	16.199	21.334	0.9	0.87	0.58
5	92.77	14.898	20.033	0.91	0.86	0.8
6	89.58	13.04	22.026	0.92	0.85	0.94
7	90.19	13.05	19.617	0.92	0.87	0.98
8	94.63	12.57	21.121	0.93	0.87	0.98
9	94.02	12.215	20.467	0.93	0.88	0.98
10	94.82	11.609	36.194	0.93	0.8	0.98

Config

Pruning with high ramp_mult (20). All weights are pruned at the end of third epoch

Epoch	Time (s)	Train Loss	Test Loss	Train Acc	Test Acc	Sparsity
1	91.27	35.411	23.284	0.69	0.84	0.04
2	96.94	26.752	29.410	0.79	0.80	0.74
3	95.56	36.676	44.136	0.66	0.50	1.00
4	87.66	44.078	43.975	0.50	0.50	1.00
5	91.71	44.293	44.319	0.50	0.50	1.00

Other

Python function used to parse output of learner to org.

To parse output paste it in this cell

---------------------------------------------------------------------------------------------------------------
| end of epoch   1 | time: 110.37s | train/valid loss 35.547/25.801 | train/valid acc 0.69/0.83 | sparsity 0.00
---------------------------------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------------------------------
| end of epoch   2 | time: 100.53s | train/valid loss 24.139/23.167 | train/valid acc 0.84/0.81 | sparsity 0.08
---------------------------------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------------------------------
| end of epoch   3 | time: 103.67s | train/valid loss 19.789/22.054 | train/valid acc 0.87/0.84 | sparsity 0.26
---------------------------------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------------------------------
| end of epoch   4 | time: 101.60s | train/valid loss 16.809/20.582 | train/valid acc 0.89/0.87 | sparsity 0.45
---------------------------------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------------------------------
| end of epoch   5 | time: 103.79s | train/valid loss 15.053/20.901 | train/valid acc 0.91/0.87 | sparsity 0.64
---------------------------------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------------------------------
| end of epoch   6 | time: 107.55s | train/valid loss 14.253/21.799 | train/valid acc 0.91/0.87 | sparsity 0.81
---------------------------------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------------------------------
| end of epoch   7 | time: 112.47s | train/valid loss 14.276/24.024 | train/valid acc 0.92/0.87 | sparsity 0.94
---------------------------------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------------------------------
| end of epoch   8 | time: 104.80s | train/valid loss 11.653/22.526 | train/valid acc 0.93/0.84 | sparsity 0.94
---------------------------------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------------------------------
| end of epoch   9 | time: 100.44s | train/valid loss 11.072/24.969 | train/valid acc 0.93/0.86 | sparsity 0.94
---------------------------------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------------------------------
| end of epoch  10 | time: 113.35s | train/valid loss 11.093/22.774 | train/valid acc 0.93/0.85 | sparsity 0.94
---------------------------------------------------------------------------------------------------------------

After that run block with org-babel-execute-src-block

from parse import parse
in_fmt = '| end of epoch {:3d} | time: {:5.2f}s ' \
         '| train/valid loss {:05.3f}/{:05.3f} ' \
         '| train/valid acc {:04.3f}/{:04.3f} | sparsity {:.2f}'

lines = list(filter(lambda line: '-'*111 not in line,  s.strip().split('\n')))
lines = list(map(lambda line: line.strip(), lines))
out_fmt = '| {} | {} | {} | {} | {} | {} | {} |\n'
res     = '| Epoch | Time (s) | Train Loss | Test Loss | Train Acc | Test Acc | Sparsity |\n' \
          '|-------+----------+------------+-----------+-----------+----------+----------|\n'
for line in list(lines):
    res += out_fmt.format(*parse(in_fmt, line))
return res

Paste this to the file and press TAB to allign