Join GitHub today
GitHub is home to over 40 million developers working together to host and review code, manage projects, and build software together.Sign up
Tried to use a Model Builder (AutoML) with a 1 GB csv file.
Out of Memory Exception
The file would get loaded. It looks like the Model Builder is trying to use a StringBuilder here. That's naturally going to fall down after a certain size of csv. Stream?
For the model builder team:
This avoids incompatibilities between the two (and it streams nicely).
@mayoatte : My apologies, my comment was more for the Model Builder team as a way to fix your reported bug. I've updated my comment to that effect.
Though, you can also use the AutoML API directly for the same reason. You can also try the ML.NET CLI: https://docs.microsoft.com/en-us/dotnet/machine-learning/how-to-guides/install-ml-net-cli
@justinormont No worries at all.
After looking at it again, i wasn't sure if my csv was in the right formt (I also have csv columns). And in case, the API is a better fit for me as i was able to reuse all the data loading and prep code I had already without going back to csv.
Please free to reopen the issue if you'd still like the Model Builder team to have a look.
Thanks again for all your help.