Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Relations should be a separate thing #117

Closed
benjie opened this issue Nov 29, 2022 · 0 comments · Fixed by #260
Closed

Relations should be a separate thing #117

benjie opened this issue Nov 29, 2022 · 0 comments · Fixed by #260
Labels

Comments

@benjie
Copy link
Member

benjie commented Nov 29, 2022

Currently relations belong to sources; however they really ought to belong to codecs (it doesn't matter where you get a row from: table, function, whatever; if it has an organization_id column then you should be able to get the organization associated with it).

However, relations can't exist until all the sources exist, so we have PgSourceBuilder as like a "this will be a PgSource, but it isn't right now" placeholder so that we can do circular relations. It's all weird and yucky.

Then at runtime, half the time we're talking about codecs, rather than sources, and they don't have relations. So currently we're doing hacky nonsense where we find a table-like source for that codec and pull the relations out of that. It's really wrong and unpleasant.

Solution: remove relations from sources; instead create them as a later phase and put then onto build.input as well - so now we get build.input.pgSources and build.input.pgRelations. Relations will relate a codec to a source - doesn't matter where the original row came from, but when fulfilling it we have to get it from somewhere (a source).

We're still going to want things like source.getRelations() to work, not 100% sure how to address that right now - perhaps pass the input object into this method...

Anyway, I'm going to keep the hacky way for now so I can keep making progress, but I'm really looking forward to getting rid of PgSourceBuilder!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant