Feat/postgres schema support

[thisisharsh7]

fixes #722

feature implemented ->

• Added schema field to spec for Postgres target
• When schema is specified, table_name must also be explicitly specified
• automatically create the schema if not exists
• it doesn't drop the schema
• targets from different flows can share the same schema
• Schema is created but not owned, users can add their own tables

[thisisharsh7]

please let me know for any changes thanks!!

[georgeh0]

Can you also help with updating this doc: https://github.com/cocoindex-io/cocoindex/blob/main/docs/docs/sources/index.md?plain=1

Thanks!

[georgeh0]

Let's leave table name unquoted for this case for now. Only double-qutes schema names.

We considered adding double-quotes for table names before, but didn't proceed because it'll break existing pipelines. We'll make this change in our next major version upgrade (e.g. v1) which is considered to be backward imcompatible.

[georgeh0]

We only need to do it when we need to run CREATE TABLE .... We can move it before executing that statement.

[thisisharsh7]

okay got it

[georgeh0]

Can you also describe how you test it end to end? Thanks!

[thisisharsh7]

Can you also describe how you test it end to end? Thanks!

thanks! I tested this feature with local postgres server:

• with both schema and table name -> it works
• with just schema no table name -> it works too and without schema at all

for database verification:

• created a schema using CREATE SCHEMA IF NOT EXISTS test_schema
• verified the schema in the database
• created a table inside the schema
• inserted and queried data successfully

[thisisharsh7]

[georgeh0]

Seems these tests happen on single statements, instead of as part of cocoindex flow?
Did you test it end to end with CocoIndex? For example, pick any existing example with Postgres target, and set the schema, and run setup and update.

[georgeh0]

I think at least the field also needs to be added to this dataclass in Python SDK:

cocoindex/python/cocoindex/targets/_engine_builtin_specs.py

Lines 12 to 16 in afea19d

	class Postgres(op.TargetSpec):
	"""Target powered by Postgres and pgvector."""
	database: AuthEntryReference[DatabaseConnectionSpec] | None = None
	table_name: str | None = None

So that users will be able to use it.

[thisisharsh7]

okay will do

[thisisharsh7]

    database: AuthEntryReference[DatabaseConnectionSpec] | None = None
    table_name: str | None = None
    schema: str | None = None

[thisisharsh7]

can you please check this if this is correct thanks!

[thisisharsh7]

I picked text-embedding example and put the schema then ran the setup nd update cmd

[georgeh0]

Can you also help with updating this doc: https://github.com/cocoindex-io/cocoindex/blob/main/docs/docs/sources/index.md?plain=1

Thanks!

Thanks! Looks good! Can you also help update the doc in the location above? Thanks!

[thisisharsh7]

yes

[thisisharsh7]

what should I include also do I need to update target file? some pointers

[georgeh0]

what should I include also do I need to update target file? some pointers

Sorry I gave the wrong link. It should be for the target, not for the source: https://github.com/cocoindex-io/cocoindex/blob/main/docs/docs/targets/postgres.md?plain=1

The description for new schema field in the spec should be quite similar to table_name.

[thisisharsh7]

please check this is this right

[georgeh0]

please check this is this right

Looks good. Please push the change. Thanks!

[georgeh0]

Looks good! Thanks for the PR. Merging.