Add MAC metric

[vlomonaco]

It would be nice to have also a MAC metric. It's difficult to compute it in native pytorch for every possible layer, but it would be a nice, hardware independed metric.

Any idea on how to do this easily in PyTorch?

[AntonioCarta]

A simple implementation can ignore the cost of activation functions and only compute the cost of feedforward and convolutional layers. Then you have to manage all the cases for convolutional layers.

The biggest problem is that for convolutional layers you need to know the input size, and therefore the layer's order.

[vlomonaco]

@AntonioCarta Maybe we can create a script that given a pytorch model:

1. Convert every conv layer into a custom conv layer which also counts MAC for it (on-the-fly during the forward pass) and saves them in a common attribute of the model object (let's say a dict).
2. Run the model on the first mini-batch of the chosen benchmark and return MAC statistics

What do you think?

[AntonioCarta]

Yes, that may be a solution. It should also be possible with forward hooks without subclassing.
I can try to implement it.

[akshitac8]

@AntonioCarta Is this issue solved?

[AntonioCarta]

Not yet. If you want you can work on this @akshitac8.

[AntonioCarta]

@akshitac8 are you working on this? It's not clear to me since I see that I am still assigned to the issue. Please give me a confirmation and assign the issue to yourself if you are taking it.
Otherwise I will work on this.

[akshitac8]

I am sorry for not giving you a clear sign on it. I had a medical emergency but now I have started working on this bust not clear just MAC metric is needed or both this issue and #60 are related?
if you can clear me on this and then you can assign me the issue :)

[vlomonaco]

@AntonioCarta any news on this? Should we disassign this issue?

[AntonioCarta]

Sorry, I forgot this issue was still assigned to me. I can do it (with the limitations discussed above), give me about a week of time to complete it.

[vlomonaco]

This may be interesting to look into when we want to upgrade the profiler: https://github.com/zhijian-liu/torchprofile