how to deal the large data when training model

[sano176]

my compute have 70G memory, my training data 2G, the text classification of model_zoo could not work.

[ljshou]

Hi @sano176 you are training on GPU right? Currently we are enabling laze load mode, which will solve your problem.

[sano176]

@ljshou ok! Looking forward to updating

[chengfx]

Hi @sano176 . The lazy-load mechanism is being reviewed in this #50 . You can test this feature using dev/fecheng branch in advance.

[sano176]

thank you very much, @chengfx , I will test it later and check if there are any problems.

[ljshou]

@sano176 have you got chance to take a try @chengfx's change?

[sano176]

It solved my problem， thank you very much!