Tree ensemble transformations

[RAMitchell]

@hcho3 I am interested in doing research on transformations such as combining multiple trees together, different pruning methods etc.

Would we be able to support these transformations as a part of the API?

Something like:

model.transform(pruning_transformation)
model.compile()
...

Having an intermediate representation of a tree structure, independent of the algorithm that generated it, is a very useful thing for this kind of work.

[hcho3]

@RAMitchell Indeed, we should plan to add back-end optimizations. Right now, treelite simply converts a tree model into hierarchical if-else statements. Tree transformations will be an essential element.

[hcho3]

@RAMitchell Sorry for the delay. I've been preoccupied with making last-minute fixes before the official launch.

I already see that the compiler portion needs some re-working; right now, it's a giant blob of script that generates C program on the fly. Really, there needs to be a good intermediate representation to enable the kind of work you and I would like to do (transformation).

Can we schedule a time for a phone or Skype call to discuss this topic further?

[hcho3]

@RAMitchell I have just started looking into model compression / pruning for decision trees. One thing we'd need to decide is a set of transformations we should support. Right now, I'm trying to reach out to potential users and customers and learn the concrete use cases of model compression for trees.

[hcho3]

Closing this for now. Right now, Treelite model object is used as an Intermediate Representation for cuML's Forest Inference Library.