Join GitHub today
GitHub is home to over 28 million developers working together to host and review code, manage projects, and build software together.Sign up
Proposal for Fluent API #474
In this issue I describe a proposal for a fluent API for the building of ML.NET learning pipelines. This API would be consistent with existing .NET patterns such as LINQ, allowing people new to ML.NET to pick it up easily. It would allow clear concise code for simple scenarios, whilst allowing easy extension for more complex situations.
The recent proposal for a major API change by @TomFinley in issue #371 is a bit step forward towards a more natural programming model, with each step of the pipeline new-ed up in turn. I would argue however that this no longer reflects the true flow through a learning pipeline, with previous steps being relegated to a parameter of the constructor. This proposal builds on top of #371 with a fluent API.
By using extension functions (in a similar manner to LINQ) we can pass the previous step of a pipeline as the 'this' parameter into subsequent steps, preserving the natural flow. For example,
This could be further cleaned up to,
More complex examples,
You could easily write extension functions that combine multiple steps, but could be consumed in the same way. Something like the following (I've created a hypothetical
You could easily merge data from two pipelines,
You could take advantage of tuples to split the data pipeline, such that different steps could be applied before later merging,
This is an outline proposal for an alternative API that could be used alongside, or instead of that proposed in issue #371. There are still some rough edges here and there, but I hope that this will start a discussion of the posibilites provided by a fluent API.
@Andy-Wilkinson, I like this idea. Adding a number of extension methods that allow for common operations is attractive.
I think that, because they represent a 'convenience layer' on top of the API, they probably should be put into a separate namespace, so that the user can 'opt in' to use them.
Not to mention that a user could create their own custom layer of extension methods on top of pipelines, tailored to the typical ML tasks that they perform.
Note that #371 actually puts a barrier against such usage (or at least calls for an extensive framework on top of direct instantiation, something we wanted to avoid), but the 'pipelines' as they currently are suggested in #581 will be compatible with these extensions.
Thanks for the comments @Zruty0. I did read #581 with interest as it fits well with the fluent style API I discussed here. I would disagree however that a fluent API would be an extensive framework on top of #371 - there would simply be some light-weight wrappers around the constructors. In fact, some of the work described in #371 regarding additional convenience constructors could equally be achieved via extension methods.
For example the
Whether to move this 'convenience layer' into a separate namespace probably depends on what the target audience is for the framework. My understanding was that this was aimed at existing .NET developers who want a straightforward on-ramp for ML tasks, hence I would argue that a convinience API should be the default (and the direction guided to by documentation, samples, etc.). Only when performing more complex tasks should developers need to delve deeper.