Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.

Already on GitHub? Sign in to your account

馃悰 Bug: NoSuchTableError is raised if user tries to access a stream that did not receive records #30

Closed
aaronsteers opened this issue Feb 12, 2024 · 0 comments 路 Fixed by airbytehq/airbyte#35311
Assignees
Labels
bug Something isn't working

Comments

@aaronsteers
Copy link
Contributor

aaronsteers commented Feb 12, 2024

Currently we only run 'create table' when and if record batches are being processed for that stream. Since we know which stream names are selected, and since we have the schema declared in the catalog for those streams, we could optionally also create empty destination tables and initialize empty CachedDatasets for those streams.

This would be helpful because a user does not expect to get an error when checking for the results of a specific stream. Example, the "deployments" stream of the GitHub source is very likely to be empty. A user iterating through the streams they selected will get a confusing failure rather than just seeing an empty dataset.

Example error from Github (internal slack thread here):

sqlalchemy.exc.NoSuchTableError: deployments
@aaronsteers aaronsteers changed the title Consider creating cache table even if no records are emitted for a given stream 馃悰 Bug: NoSuchTableError is raised if user tries to access a stream that did not receive records Feb 12, 2024
@aaronsteers aaronsteers added the bug Something isn't working label Feb 12, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
2 participants