Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Any option to define destination db's schema? #1968

Open
saytesnake opened this issue May 20, 2024 · 3 comments
Open

Any option to define destination db's schema? #1968

saytesnake opened this issue May 20, 2024 · 3 comments
Labels
enhancement New feature or request

Comments

@saytesnake
Copy link

Is your feature request related to a problem?

Well, in some dev env, sometimes the db's schema is different, not just "public".

Describe the solution you'd like

Just like source db's public.aaa1 sync to destination db's abc.aaa1。

Describe alternatives you've considered

Table to table with mapping, just like source db's public.aaa1 sync to destination db's public.bbb1? Not sure if it's a good idea.

Thx !

@saytesnake saytesnake added the enhancement New feature or request label May 20, 2024
@nickzelei
Copy link
Member

Hi there! Thanks for the feature request.
This is something we have considered. It's not on our immediate roadmap, but is something we'd like to see added to Neosync eventually!

There are definitely a few different usecases here that I can think of:

  1. Schema Migration. (highlighted above). -- When you want to move all or some of your tables from one schema to another.
  2. Table Migration. -- When you want to move your data from one table to another

@nickzelei
Copy link
Member

nickzelei commented May 20, 2024

@saytesnake Can you tell me more about your immediate need/usecase?
Is it that in production/some env your tables are in X schema but in development you have them in Y schema?

@saytesnake
Copy link
Author

@saytesnake Can you tell me more about your immediate need/usecase? Is it that in production/some env your tables are in X schema but in development you have them in Y schema?

I have a usecase as follows, to synchronize tables from a specific database across multiple PostgreSQL instances into one database in a single PostgreSQL instance, using schema to distinguish each database. This is for simple offline analysis, statistics, etc., and to create views for other vendors to access and use.

I know some extension like postgres_fdw and dblink, but the performance is not ideal, especially when using views (although this might be related to the quality of the SQL queries).

Some tools, such as Airbyte, is too complicated for me, and maybe I don't need the big data tech stack (such as ETL, ODS, DW...). Therefore, we found neosync on Github, which is an excellent tool that is simple and easy to use.

graph TD;
	PG_1.public.aaa1-->neosync-->PG_AIO;
	PG_2.public.bbb1-->neosync-->PG_AIO;
	PG_3.public.ccc1-->neosync-->PG_AIO;
        PG_AIO-->PG_AIO.pg_1.aaa1;
        PG_AIO-->PG_AIO.pg_2.bbb1;
        PG_AIO-->PG_AIO.pg_3.ccc1;

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants