Join GitHub today
GitHub is home to over 50 million developers working together to host and review code, manage projects, and build software together.Sign up
Streaming API based ItemWriter for JSON, with restart support [BATCH-1850] #1739
Since JSON has become a widely adapted format among Integration technologies, a Streaming API based ItemWriter with restart support for JSON has to be developed. On the similar lines to StaxEventItemWriter.
Writing a simple Itemwriter wouldnt suffice. As in most batch process the datasets could be enormous,so the job should support restarts.
If the above has to be achieved with XML as the output format, it can be easily achieved using StaxEventItemWriter.
A similar implementation for JSON would be highly valuable.
2 votes, 3 watchers
Dave Syer commented
I didn't get the RabbitMQ piece (5), but can't you do everything with a FlatFileItemWriter plus a custom LineAggregator (and maybe header/footer callbacks to complete the JSON object)? We could definietly add one of those (to complement the existing JsonLineMapper).
Vasanth Raj commented
RabbitMQ part isnt related to the issue, but mentioned it for use case to look complete, end to end. the output file needs to be placed in a Queue, probably a JobListener should do the job.
Using the technique you suggested above should work for simple scenarios, but letting a parser do the work would make the solution more accommodating for future changes.
eg: If there will be 2 item processors writing 2 different fragments to the same file ??