Truely enjoy Smile, just saying Hi

[JayvanBruggen]

I didn't see a lot of discussion on StackExchange [smile] or here,
but I do want to mention that I am working hard every day on a private project (a trading algo) that is implementing parts of Smile.
I am just very happy that it exists, and want to say thank you to the team building this, it is amazing what is all in there.
Java Is my main language, The past year I learned first steps of ML on Python using scikit learn, and then moved to Smile on Java when I could, to continue my ML course there, now having a grasp on what is what. So I build the Zalando clothing test set classifier and was amazed. Awesome. Just as easy as scikit learn.
Some minor issues: Why isn't it easy to delete rows in a DataFrame? I could try setting nulls in fields and then remove rows with nulls, but, mwhah. So I removed stuff before I put it in the DataFrame.
There is some train test split functionality in Smile, but it takes arrays []. The documentation seems rather proud announcing there are Dataframes so I programmed towards DataFrames (they work well by the way). Now to my surprise I would have to unpack them again for the splits , I was surprised and I won't. I'm splitting the Dataframes myself. So is that older code? I bet older smile was only using arrays and that is where that came from?
Anyway, greets, Thanks for the work,
Jay

[haifengl]

Thanks Jay! It is a slow operation to delete rows one by one. A better approach is to return a new DataFrame with get(boolean[]), get(Index), or by stream processing

var stream = iris.stream().filter(row -> row.getDouble(1) > 3 && row.getByte("class") != 0)
var filtered = DataFrame.of(iris.schema(), stream)

It is similar with data split. For generality, the data splitting functionalities such as cross validation, bootstrap return a set of Bags, which are sample index for training and testing, respectively. You can use them to create new data frames:

var train = df.get(Index.of(bag.samples()));
var test = df.get(Index.of(bag.oob()));

[JayvanBruggen]

Thank you haifengl, I'll check it out and see if I will use it. Of course there are many paths, and I made something that works in my case, even with speed in mind. Thanks for your response, if I run into big difficulties, I'll come here to discuss. Greets.

[haifengl]

I add a few convenient functions in Bag class for data splitting such as split and stratify. You can apply them to DataFrame directly. For more advanced use cases, I suggest Bootstrap and CrossValidation classes. Please let me know if you have questions or suggestions to the new functions.