Clover overhead in inverter ~50%

[kostrzewa]

Okay, so I went ahead and did the long overdue measurement of the overhead of using the clover term in the inverter.

The result is that 50% of the time spent in the operator is spent in the hopping matrix, 25% in clover_inv and 25% in clover_gamma5.

As a comparison, without the clover term 50% of time is spent in H_eo_tm_inv_psi and 50% of time is spent tm_sub_H_eo_gamma5.

Keeping this in mind I'm a bit surprised that with the clover term the performance reported in the inverter, not taking into account the clover overhead, is about 50-60% lower than for the standard operator...

Unfortunately I can't really give a recommendation for improving the situation.

[kostrzewa]

Actually, I think I know one sure-fire way of improving performance. If we prepare a copy of the sw field with the correct ordering (even-odd, I guess), this should help quite a bit. Other than that I'm not sure what to do.

[ggscorzato]

Hi Bartek,
can you please show me the directory where you did these tests?
Maybe I can start from there and take more measurements, with the help of the tuning experts.

[kostrzewa]

Hi Luigi, the scalasca output is in:

/work/hch02/hch028/invert_test/invert_test_hybrid_hs_scalasca_512_64/invert_test_clover
/work/hch02/hch028/invert_test/invert_test_hybrid_hs_scalasca_512_64/invert_test_no_clover

The inversions of course have differing iteration numbers so only relative weights between the "hot-spots" are meaningful.

[ggscorzato]

thanks Bartek!

can you give me access to these directories?

[kostrzewa]

They should be world-readable, at least according to the unix permissions.

[ggscorzato]

I get:

[hch02b@juqueen2 hch02]$ cd /work/hch02/hch028/invert_test/invert_test_hybrid_hs_scalasca_512_64/invert_test_clover
/work/hch02/hch028/invert_test/invert_test_hybrid_hs_scalasca_512_64/invert_test_clover: Permission denied.
[hch02b@juqueen2 hch02]$ cd /work/hch02/hch028/invert_test/invert_test_hybrid_hs_scalasca_512_64/
/work/hch02/hch028/invert_test/invert_test_hybrid_hs_scalasca_512_64/: Permission denied.
[hch02b@juqueen2 hch02]$ more /work/hch02/hch028/invert_test/invert_test_hybrid_hs_scalasca_512_64/invert_test_clover
/work/hch02/hch028/invert_test/invert_test_hybrid_hs_scalasca_512_64/invert_test_clover: Permission denied

[kostrzewa]

oh, the grand-grand-parent directory had the wrong permissions, it should be fixed now

[ggscorzato]

yes!
thanks!