Usage for large excel files

[architpuri]

Feature 7
DB Batch Insertion

I have a use case where I want to dump data to DB but in batches eg read 1500 rows and insert in async. How can I create such a consumer?

[ozlerhakan]

Hi @architpuri ,

Have you looked at the example 7 https://github.com/ozlerhakan/poiji#feature-7 , maybe you can iterate your data in a number batch and use this option. wdyt?

[ozlerhakan]

Ah you want a bulk insert, am I right?

[architpuri]

Yes

[ozlerhakan]

you should do it by yourself since we provide a list from Poiji.fromExcel we return a list that you can infer it as a batch or you can add each row into a list using the example 7 and track the row number using @ExcelRow so that when reaching a certain row number, you can insert them as a bulk data.

[architpuri]

So you are basically saying that I can maintain a global list and use it like this.

List<Calculation> accumulate = new ArrayList();
Poiji.fromExcel(fileCalculation, Calculation.class, options, this::dbInsertion);

private void dbInsertion(Calculation siCalculation) {
  if (accumulate.size() >1000){
   List<Calculation> insertList = new ArrayList();
   insertList.addAll(accumulate);
   insertDBAsyncCall(insertList);
   accumulate = new ArrayList();
  }
 accumulate.add(siCalculation);
}

[ozlerhakan]

have you tried your approach @architpuri ?