You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Users currently cannot select which fields (columns) they want to sync from their streams -- if you pick a stream, it's all or nothing. However, users want the ability to project which fields they want. A simple use case is sensitive data like described in #2463 . We can do this at the Airbyte level without having to do it separately for every connector by filtering the incoming records to contain only the selected fields in the catalog.
Sources can still implement projection as needed (e.g: a DB source will still want to select only the relevant columns for efficiency reasons), but this can be implemented "for free" outside the connectors.
Describe the alternative you’ve considered or used
Projection at the destination level
The text was updated successfully, but these errors were encountered:
Tell us about the problem you're trying to solve
Users currently cannot select which fields (columns) they want to sync from their streams -- if you pick a stream, it's all or nothing. However, users want the ability to project which fields they want. A simple use case is sensitive data like described in #2463 . We can do this at the Airbyte level without having to do it separately for every connector by filtering the incoming records to contain only the selected fields in the catalog.
Sources can still implement projection as needed (e.g: a DB source will still want to
select
only the relevant columns for efficiency reasons), but this can be implemented "for free" outside the connectors.Describe the alternative you’ve considered or used
Projection at the destination level
The text was updated successfully, but these errors were encountered: