Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Not all Source messages should be sent to the Destination #16417

Closed
evantahler opened this issue Sep 7, 2022 · 0 comments · Fixed by #16418
Closed

Not all Source messages should be sent to the Destination #16417

evantahler opened this issue Sep 7, 2022 · 0 comments · Fixed by #16418
Assignees

Comments

@evantahler
Copy link
Contributor

evantahler commented Sep 7, 2022

Consider this code within the Replication Worker:

final AirbyteMessage airbyteMessage = messageOptional.get();
validateSchema(recordSchemaValidator, validationErrors, airbyteMessage);
final AirbyteMessage message = mapper.mapMessage(airbyteMessage);
messageTracker.acceptFromSource(message);
try {
destination.accept(message);
} catch (final Exception e) {
throw new DestinationException("Destination process message delivery failed", e);
}

I /think/ this means that all messages the worker receives from the source are then passed on to the destination. However, only RECORD and STATE messages should be sent to the destination, ignoring LOG and TRACE messages.

@evantahler evantahler changed the title Not all source messages should be sent to the Destination Not all Source messages should be sent to the Destination Sep 7, 2022
@evantahler evantahler self-assigned this Sep 8, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant