memory leak

[wassname]

Thanks for sharing, this is a simple and interesting way to use auxiliary losses.

When using it on a large dataset I get a memory leak, it uses up more and more cuda memory untill it crashes. I think this is because the graph is not cleared loss.backward(retain_graph=True).

The obvious next step is to clear the graph with a loss.backward(retain_graph=False) but I get an error that the variables have been modified (image blow). I assume this is an intentional or metabalance, but I can't find where, and I can't find a way to clear the graph manually.

Any tips?

[wassname]

I'm going to close this, because this issue seems more complicated than I originally thought. Perhaps it's my torch version or I do have an in-place operation. It's really quite hard to pin down into a clear question I'm afraid.