colab notebook runtime disconnects when numdata_points = 20000

[ln0v]

I started from the demo income classification , using the linear regressor - I am using my own data set to predict insurance payments using about 12 features. My dataset has over 30,000 points, the what-if tool disconnect when I set the number of data points to 20000. whatif works up to about 15,000 numdata_points.
This is not a time out problem, code runs for under 5 minutes. Is there a limit on the number of data points that WitConfigBuilder can handle in colab.
response would be much appreciated.

[jameswex]

The limit seems to be based on the number of examples, the size of examples, and possibly also on your browser/computer as well. We're currently doing a deep dive into what causes WIT failures (including colab runtime failures when running in colab), investigating the intersection of all of those variables. At the end of that, we should be able to provide more documentation/guidance, and possibly also find some fixes we can make to raise the limit and/or provide warnings.

In general, there are failures caused by the colab kernel crashing due to its own memory issues, and also failures caused by the browser tab's allowed memory being exceeded.

Your experience of having issues at around 15k examples matches up with our experience.

For now I would recommend sampling your data to keep the datapoint count around 10k, and we should have more data/information on WIT memory/runtime issues in the upcoming weeks.