FastTree uses lots of memory with dropout on

[daholste]

System information

• OS version/distro: Windows 10
• .NET Version (eg., dotnet --info): .NET Core 2.1

Issue

• What did you do?
  Ran FastTreeRegressor with DropoutRate > 0.

• What happened?
  Consumed around 15 GB of memory. Memory usage increased linearly with # of trees

Running FastTreeRegressor with DropoutRate > 0 leads to much higher memory consumption than when DropoutRate = 0. It used approx 14 GB more for 200 trees, and ran into an out of memory exception for 500 trees. Perhaps there is a leak? (Turning on DropoutRate seemed to consistently cause memory footprint to rise significantly.)

[TomFinley]

I don't think it's a leak. It achieves dropout by memorizing each score per document, per tree. So, not really a "bug" per se. Not great, but expected and not a bug. We could potentially think about reducing the memory usage somehow, but such an optimization would have to be weighed against how valuable DART is, really, in the context of FastTree specifically.

[daholste]

really interesting -- thank you, Tom!

[shauheen]

Closing this issue. @daholste please feel free to reopen if you have further questions.