Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Handle fault tolerance delivery guarantees for child documents #177

Open
nsoft opened this issue Mar 18, 2023 · 0 comments
Open

Handle fault tolerance delivery guarantees for child documents #177

nsoft opened this issue Mar 18, 2023 · 0 comments
Labels
Milestone

Comments

@nsoft
Copy link
Owner

nsoft commented Mar 18, 2023

As it stands (and as it will be for 1.0) the Fault tolerance implementation attempts to ensure at most once delivery in a manner that approaches only once delivery. Failure to deliver should be limited to cases where the system happens to have a power-cord style failure between the recording of send and the actual send of the document.

In it's current form it does not properly account for child document creation however, so any child documents generated within a Processor (any case where the processor returns more than one document) may re-attempt all children if any children have failed.

One possible work around for this would be to write the children to disk (java serialization), and then re-read them with a filescanner. The proper fix for this will be to identify child producing processors and treat them as if they were scanners. (and migrate terminology to 'document source' vs 'scanner')

Calling this a bug, but it will be a known issue for 1.0 since many use cases don't involve producing child documents and the system should be perfectly functional for any other use case.

@nsoft nsoft added the bug label Mar 18, 2023
@nsoft nsoft added this to the 1.1 milestone Mar 18, 2023
@nsoft nsoft modified the milestones: 1.1, 1.2 Apr 27, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

1 participant