Join GitHub today
GitHub is home to over 36 million developers working together to host and review code, manage projects, and build software together.Sign up
Use lazy loading instead of one-off loading #28
Hi, @ljshou . I have tested Query Binary Classification task using 35 million data( The size of file is about 1.5G) In my local PC( Memory 32G) already. It can run normally, including building dictionary, encoding raw data and training.
For 1000 million data, I think current version could build dictionary, but will fail at encoding raw data.