dynamic composite reader/processor/writer [BATCH-2644]

[spring-projects-issues]

ohad opened BATCH-2644 and commented

Currently as far as I understand, there is no option in Spring Batch to have a dynamic composite reader/processor/writer

The idea is to have the ability to replace processor at runtime, and in case of multiple processors (AKA composite-processor), to have the option to add/remove/replace/change order of processors. As mentioned, same for reader/writer.

I thought of something like reading the processors list from DB (using cache?) and there the items (beans' names) can be changed. Does this make sense?

why is this required?

There are cases that I use processors as "filters", and it may occur that the business (the client) may change the requirements (yes, it is very annoying) and ask to switch among filters (change the priority).
Other use case is having multiple readers to get the data from different data warehouse, and again - the client changes the warehouse from time to time (integration phase), and I do not want my app to be restarted each and every time.
There are many other use cases, of course. (other cases can be found in StackOverflow, where people ask for this feature) .

---

Affects: 4.0.0.M3

Reference URL: https://stackoverflow.com/questions/46695238/spring-batch-dynamic-composite-reader-processor-writer

[spring-projects-issues]

Michael Minella commented

Why is a step scoped FactoryBean not good enough for this use case?

[spring-projects-issues]

ohad commented

not sure I follow... FactoryBean is an interface; is there an impl in spring-batch that reads dynamically the list of processors from DB (for example)?
the idea is not to have a static list of processors, but to loads it from DB so if i wanna change this list in runtime (or change the order) - i will be able to do so.

[spring-projects-issues]

Michael Minella commented

You'd have to implement it. It would be a factory bean that read from a db what you wanted and formed the beans as needed. I'm not sure this is so generic that a new component for the framework makes sense but I believe writing your own would work.

[spring-projects-issues]

ohad commented

cool. I've started working on it here:

https://github.com/OhadR/spring-batch-dynamic-composite

i'd be more than happy to integrate it into spring-batch.

[spring-projects-issues]

ohad commented

@Michael Minela
you closed this item, it means that you do not want me to work on it and to prepare a merge request ?