Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

dynamic composite reader/processor/writer [BATCH-2644] #959

Closed
spring-issuemaster opened this issue Oct 14, 2017 · 5 comments
Closed

dynamic composite reader/processor/writer [BATCH-2644] #959

spring-issuemaster opened this issue Oct 14, 2017 · 5 comments

Comments

@spring-issuemaster
Copy link
Collaborator

@spring-issuemaster spring-issuemaster commented Oct 14, 2017

ohad opened BATCH-2644 and commented

Currently as far as I understand, there is no option in Spring Batch to have a dynamic composite reader/processor/writer

The idea is to have the ability to replace processor at runtime, and in case of multiple processors (AKA composite-processor), to have the option to add/remove/replace/change order of processors. As mentioned, same for reader/writer.

I thought of something like reading the processors list from DB (using cache?) and there the items (beans' names) can be changed. Does this make sense?

why is this required?

There are cases that I use processors as "filters", and it may occur that the business (the client) may change the requirements (yes, it is very annoying) and ask to switch among filters (change the priority).
Other use case is having multiple readers to get the data from different data warehouse, and again - the client changes the warehouse from time to time (integration phase), and I do not want my app to be restarted each and every time.
There are many other use cases, of course. (other cases can be found in StackOverflow, where people ask for this feature) .


Affects: 4.0.0.M3

Reference URL: https://stackoverflow.com/questions/46695238/spring-batch-dynamic-composite-reader-processor-writer

@spring-issuemaster

This comment has been minimized.

Copy link
Collaborator Author

@spring-issuemaster spring-issuemaster commented Oct 16, 2017

Michael Minella commented

Why is a step scoped FactoryBean not good enough for this use case?

@spring-issuemaster

This comment has been minimized.

Copy link
Collaborator Author

@spring-issuemaster spring-issuemaster commented Oct 16, 2017

ohad commented

not sure I follow... FactoryBean is an interface; is there an impl in spring-batch that reads dynamically the list of processors from DB (for example)?
the idea is not to have a static list of processors, but to loads it from DB so if i wanna change this list in runtime (or change the order) - i will be able to do so.

@spring-issuemaster

This comment has been minimized.

Copy link
Collaborator Author

@spring-issuemaster spring-issuemaster commented Oct 16, 2017

Michael Minella commented

You'd have to implement it. It would be a factory bean that read from a db what you wanted and formed the beans as needed. I'm not sure this is so generic that a new component for the framework makes sense but I believe writing your own would work.

@spring-issuemaster

This comment has been minimized.

Copy link
Collaborator Author

@spring-issuemaster spring-issuemaster commented Oct 16, 2017

ohad commented

cool. I've started working on it here:

https://github.com/OhadR/spring-batch-dynamic-composite

i'd be more than happy to integrate it into spring-batch.

@spring-issuemaster

This comment has been minimized.

Copy link
Collaborator Author

@spring-issuemaster spring-issuemaster commented Oct 17, 2017

ohad commented

@Michael Minela
you closed this item, it means that you do not want me to work on it and to prepare a merge request ?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
1 participant
You can’t perform that action at this time.