String value of datasource name with blocking queue reader

[Kmouille]

Hi Mahmoud,

First of all, thanks for this project. I like it a lot.

Following principles of tutorial "3. Dispatching records", I found out that the data source name in the job parameter is set as a String when job launches.

In my case I have a master job and 2 sub jobs reading a blokingQueue. If I launch the 3 jobs with a thread pool size of 2 then, the 3rd job (subjob 2) launches with thousands of elements in the queue.

Unfortunately, the data source name is set there with the current content of the blocking queue. This lists all the elements contained in that queue (thousands). This is a big string to store, and printing it out in the reports is very hard (that's not a readable report anymore ;-) )
Maybe the size of the queue at the beginning of the job could be enough ? or the size and the first n elements...

[benas]

Hi @Kmouille Thank you, I'm glad you like Easy Batch 😄

Thank you for pointing out this issue! Indeed, not very convenient to dump the queue content in the job report.. The BlockingQueueRecordReader uses the toString method as the data source name, which prints the queue content by default.

Maybe the size of the queue at the beginning of the job could be enough ? or the size and the first n elements...

The size of the queue would be redundant with the total records count in the job report. I suggest to simply put "In-Memory Queue" just like (and to be consistent with) the StringRecordReader (which avoids printing the whole string in the report). What do you think?

Kind regards
Mahmoud

[benas]

Hi,

I've updated the BlockingQueueRecordReader to print In-Memory Queue as data source name. This is to be consistent with the StringRecordReader and IterableRecordReader (which respectively print In-Memory String and In-Memory Iterable)

With this fix, elements in the queue will not be dumped in the job report.

Kind regards
Mahmoud

[benas]

Hi @Kmouille

The fix has been merged in the master branch and will be released in the upcoming version soon.

Regards
Mahmoud