feat(indexer): postgres schema creation + CI config #20701
Conversation
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 6
Outside diff range and nitpick comments (10)
indexer/postgres/create_table.go (2)
11-24: Ensure proper logging and error handling.The
CreateTablefunction correctly logs and handles errors. However, consider adding more context to the error messages for better debugging.if err != nil { return fmt.Errorf("failed to create table SQL: %w", err) }
77-77: Improve primary key definition.Ensure that the primary key definition is correct and consider adding more context to the error message.
_, err = fmt.Fprintf(writer, "PRIMARY KEY (%s)", strings.Join(pKeys, ", ")) if err != nil { return fmt.Errorf("failed to define primary key: %w", err) }indexer/postgres/tests/init_schema_test.go (1)
34-66: Improve error message context in tests.The
testInitSchemafunction logs SQL statements and parameters. Ensure that the error messages provide enough context for debugging.require.NoError(t, err, fmt.Sprintf("failed to log message: %s", msg))indexer/postgres/enum.go (3)
13-39: Ensure proper error handling and logging.The
CreateEnumTypefunction checks if the enum type exists and creates it if it doesn't. Ensure that the error messages provide enough context for debugging.return fmt.Errorf("failed to check if enum type %q exists: %w", typeName, err)
41-63: Ensure SQL statement generation is correct.The
CreateEnumTypeSqlfunction generates a SQL statement for creating an enum type. Ensure that the generated SQL is correct and consider adding more context to the error messages.return fmt.Errorf("failed to generate CREATE TYPE statement: %w", err)
70-91: Ensure proper handling of enum types.The
createEnumTypesForFieldsfunction creates enum types for fields with enum kind. Ensure that the function correctly handles all cases and consider adding more context to the error messages.return fmt.Errorf("failed to create enum type for field %q: %w", field.Name, err)indexer/postgres/column.go (4)
10-52: Ensure proper handling of column definitions.The
createColumnDefinitionfunction writes a column definition within a CREATE TABLE statement. Ensure that the function correctly handles all cases and consider adding more context to the error messages.return fmt.Errorf("failed to write column definition for field %q: %w", field.Name, err)
54-63: Ensure proper handling of nullability.The
writeNullabilityfunction writes column nullability. Ensure that the function correctly handles all cases and consider adding more context to the error messages.return fmt.Errorf("failed to write nullability for column: %w", err)
65-108: Ensure proper handling of simple column types.The
simpleColumnTypefunction returns the PostgreSQL column type for simple types. Ensure that the function correctly handles all cases and consider adding more context to the error messages.return fmt.Errorf("unexpected kind: %v", kind)
110-119: Ensure proper handling of updatable column names.The
updatableColumnNamefunction returns the name of the insertable/updatable column name for the field. Ensure that the function correctly handles all cases and consider adding more context to the error messages.return fmt.Errorf("failed to get updatable column name for field %q: %w", field.Name, err)
Review details
Configuration used: .coderabbit.yml
Review profile: CHILL
Files ignored due to path filters (2)
indexer/postgres/go.sumis excluded by!**/*.sumindexer/postgres/tests/go.sumis excluded by!**/*.sum
Files selected for processing (26)
- .github/dependabot.yml (1 hunks)
- .github/pr_labeler.yml (1 hunks)
- .github/workflows/test.yml (1 hunks)
- go.work.example (1 hunks)
- indexer/postgres/CHANGELOG.md (1 hunks)
- indexer/postgres/README.md (1 hunks)
- indexer/postgres/base_sql.go (1 hunks)
- indexer/postgres/column.go (1 hunks)
- indexer/postgres/conn.go (1 hunks)
- indexer/postgres/create_table.go (1 hunks)
- indexer/postgres/create_table_test.go (1 hunks)
- indexer/postgres/enum.go (1 hunks)
- indexer/postgres/enum_test.go (1 hunks)
- indexer/postgres/go.mod (1 hunks)
- indexer/postgres/indexer.go (1 hunks)
- indexer/postgres/internal/testdata/example_schema.go (1 hunks)
- indexer/postgres/module_mgr.go (1 hunks)
- indexer/postgres/options.go (1 hunks)
- indexer/postgres/sonar-project.properties (1 hunks)
- indexer/postgres/table_mgr.go (1 hunks)
- indexer/postgres/tests/README.md (1 hunks)
- indexer/postgres/tests/go.mod (1 hunks)
- indexer/postgres/tests/init_schema_test.go (1 hunks)
- indexer/postgres/tests/testdata/init_schema.txt (1 hunks)
- indexer/postgres/tests/testdata/init_schema_no_retain_delete.txt (1 hunks)
- schema/logutil/logger.go (1 hunks)
Files skipped from review due to trivial changes (8)
- .github/dependabot.yml
- .github/pr_labeler.yml
- go.work.example
- indexer/postgres/CHANGELOG.md
- indexer/postgres/base_sql.go
- indexer/postgres/enum_test.go
- indexer/postgres/go.mod
- indexer/postgres/tests/README.md
Additional context used
Path-based instructions (13)
indexer/postgres/options.go (1)
Pattern
**/*.go: Review the Golang code for conformity with the Uber Golang style guide, highlighting any deviations.indexer/postgres/conn.go (1)
Pattern
**/*.go: Review the Golang code for conformity with the Uber Golang style guide, highlighting any deviations.indexer/postgres/table_mgr.go (1)
Pattern
**/*.go: Review the Golang code for conformity with the Uber Golang style guide, highlighting any deviations.schema/logutil/logger.go (1)
Pattern
**/*.go: Review the Golang code for conformity with the Uber Golang style guide, highlighting any deviations.indexer/postgres/internal/testdata/example_schema.go (1)
Pattern
**/*.go: Review the Golang code for conformity with the Uber Golang style guide, highlighting any deviations.indexer/postgres/create_table_test.go (2)
Pattern
**/*.go: Review the Golang code for conformity with the Uber Golang style guide, highlighting any deviations.
Pattern
**/*_test.go: "Assess the unit test code assessing sufficient code coverage for the changes associated in the pull request"indexer/postgres/module_mgr.go (1)
Pattern
**/*.go: Review the Golang code for conformity with the Uber Golang style guide, highlighting any deviations.indexer/postgres/indexer.go (1)
Pattern
**/*.go: Review the Golang code for conformity with the Uber Golang style guide, highlighting any deviations.indexer/postgres/create_table.go (1)
Pattern
**/*.go: Review the Golang code for conformity with the Uber Golang style guide, highlighting any deviations.indexer/postgres/tests/init_schema_test.go (2)
Pattern
**/*.go: Review the Golang code for conformity with the Uber Golang style guide, highlighting any deviations.
Pattern
**/*_test.go: "Assess the unit test code assessing sufficient code coverage for the changes associated in the pull request"indexer/postgres/enum.go (1)
Pattern
**/*.go: Review the Golang code for conformity with the Uber Golang style guide, highlighting any deviations.indexer/postgres/column.go (1)
Pattern
**/*.go: Review the Golang code for conformity with the Uber Golang style guide, highlighting any deviations.indexer/postgres/README.md (1)
Pattern
**/*.md: "Assess the documentation for misspellings, grammatical errors, missing documentation and correctness"
LanguageTool
indexer/postgres/tests/testdata/init_schema_no_retain_delete.txt
[style] ~22-~22: The adverb ‘ALWAYS’ is usually put before the verb ‘GENERATED’.
Context: ..." BOOLEAN NOT NULL, "time" TIMESTAMPTZ GENERATED ALWAYS AS (nanos_to_timestamptz("time_nanos"))...(ADVERB_WORD_ORDER)
[typographical] ~26-~26: Consider adding a comma here.
Context: ...oat32" REAL NOT NULL, "float64" DOUBLE PRECISION NOT NULL, "bech32address" TEXT NOT NUL...(NN_NOT_NN_COMMA)
indexer/postgres/tests/testdata/init_schema.txt
[style] ~22-~22: The adverb ‘ALWAYS’ is usually put before the verb ‘GENERATED’.
Context: ..." BOOLEAN NOT NULL, "time" TIMESTAMPTZ GENERATED ALWAYS AS (nanos_to_timestamptz("time_nanos"))...(ADVERB_WORD_ORDER)
[typographical] ~26-~26: Consider adding a comma here.
Context: ...oat32" REAL NOT NULL, "float64" DOUBLE PRECISION NOT NULL, "bech32address" TEXT NOT NUL...(NN_NOT_NN_COMMA)
indexer/postgres/README.md
[uncategorized] ~3-~3: You might be missing the article “the” here.
Context: ... The PostgreSQL indexer can fully index current state for all modules that implement `c...(AI_EN_LECTOR_MISSING_DETERMINER_THE)
[misspelling] ~37-~37: This word is normally spelled as one.
Context: ... one with the_nanossuffix with full nano-seconds precision, and another as a `TIMESTAMPT...(EN_COMPOUNDS_NANO_SECONDS)
[duplication] ~39-~39: Possible typo: you repeated a word
Context: ...ype is created for each module prefixed with with module name it pertains to ...(ENGLISH_WORD_REPEAT_RULE)
Additional comments not posted (38)
indexer/postgres/options.go (1)
1-10: Ensure field names and documentation are clear.The
ManagerOptionsstruct is well-documented and the field names are descriptive. The changes align with the Uber Golang style guide.indexer/postgres/conn.go (1)
1-14: Ensure interface methods are well-documented and named appropriately.The
DBConninterface is well-defined, and the method names are descriptive. The changes align with the Uber Golang style guide.indexer/postgres/sonar-project.properties (1)
1-16: Ensure SonarCloud properties are correctly configured.The SonarCloud properties are well-configured and align with best practices for CI integration.
indexer/postgres/table_mgr.go (1)
1-44: Ensure struct and methods are well-documented and named appropriately.The
TableManagerstruct and its methods are well-defined, and the method names are descriptive. The changes align with the Uber Golang style guide.indexer/postgres/tests/go.mod (4)
1-1: Module declaration looks good.The module path is correctly specified.
3-11: Required dependencies look good.The dependencies are correctly listed and necessary for testing.
13-29: Indirect dependencies look good.The indirect dependencies are correctly listed and necessary.
31-33: Replace directives look good.The replace directives are correctly specified for local development and testing.
schema/logutil/logger.go (3)
1-3: Package declaration looks good.The package is correctly specified as
logutil.
5-22: Logger interface looks good.The interface methods are well-defined and follow best practices.
24-35: NoopLogger implementation looks good.The
NoopLoggercorrectly implements theLoggerinterface with no-op methods.indexer/postgres/tests/testdata/init_schema_no_retain_delete.txt (3)
1-5: Enum type creation looks good.The SQL statements for creating enum types
test_my_enumandtest_vote_typeare correct.
34-40: Table creation fortest_singletonlooks good.The SQL statements for creating the table
test_singletonare correct.
42-48: Table creation fortest_votelooks good.The SQL statements for creating the table
test_voteare correct.indexer/postgres/tests/testdata/init_schema.txt (3)
1-5: Enum type creation looks good.The SQL statements for creating enum types
test_my_enumandtest_vote_typeare correct.
34-40: Table creation fortest_singletonlooks good.The SQL statements for creating the table
test_singletonare correct.
42-49: Table creation fortest_votelooks good.The SQL statements for creating the table
test_voteare correct.indexer/postgres/internal/testdata/example_schema.go (6)
7-18: Ensure correct initialization ofAllKindsObject.The
AllKindsObjectis correctly initialized with a name and key fields. The initialization logic appears correct.
20-35: Verify field initialization logic.The loop correctly initializes fields for each valid kind, with special handling for
EnumKindandBech32AddressKind. Ensure thatMyEnumand address prefix are correctly defined.
37-43: Ensure correct initialization ofExampleSchema.The
ExampleSchemais initialized with object types. The initialization logic appears correct.
46-57: Ensure correct definition ofSingletonObject.The
SingletonObjectis correctly defined with a name and value fields. The definition logic appears correct.
60-83: Ensure correct definition ofRetainDeleteObject.The
RetainDeleteObjectis correctly defined with a name, key fields, value fields, and theRetainDeletionsflag. The definition logic appears correct.
85-88: Ensure correct definition ofMyEnum.The
MyEnumis correctly defined with a name and values. The definition logic appears correct.indexer/postgres/create_table_test.go (3)
10-38: Ensure correct table creation forAllKindsObject.The example function correctly creates a table for
AllKindsObjectand verifies the output. The logic appears correct.
40-49: Ensure correct table creation forSingletonObject.The example function correctly creates a table for
SingletonObjectand verifies the output. The logic appears correct.
51-57: Ensure correct helper function for table creation.The helper function correctly initializes the
TableManagerand creates the table. The logic appears correct.indexer/postgres/module_mgr.go (4)
10-17: Ensure correct definition ofModuleManager.The
ModuleManageris correctly defined with necessary fields. The definition logic appears correct.
19-27: Ensure correct initialization ofModuleManager.The function correctly initializes the
ModuleManagerwith the provided schema and options. The logic appears correct.
30-57: Ensure correct schema initialization.The method correctly initializes the schema by creating enum types and tables. The logic appears correct.
59-62: Ensure correct table retrieval.The method correctly returns the table managers for the module. The logic appears correct.
indexer/postgres/indexer.go (3)
11-20: Ensure correct definition ofConfig.The
Configstruct is correctly defined with necessary fields. The definition logic appears correct.
22-22: Ensure correct definition ofSqlLogger.The
SqlLoggertype is correctly defined. The definition logic appears correct.
24-79: Ensure correct initialization of the indexer.The
StartIndexerfunction correctly initializes the PostgreSQL indexer with the provided configuration and logger. The logic appears correct.indexer/postgres/create_table.go (2)
55-61: Optimize condition check for retaining deletions.The condition check for retaining deletions is clear and concise. Ensure that
tm.options.DisableRetainDeletionsandtm.typ.RetainDeletionsare correctly set.
87-93: Ensure proper permissions are set.The code grants SELECT permissions to PUBLIC. Ensure this aligns with security requirements and does not expose sensitive data.
indexer/postgres/tests/init_schema_test.go (2)
24-32: Ensure comprehensive test coverage.The
TestInitSchemafunction includes two test cases. Ensure that all edge cases are covered and consider adding more test cases if necessary.
68-87: Ensure proper cleanup in tests.The
createTestDBfunction creates a temporary directory and starts an embedded PostgreSQL instance. Ensure that the cleanup operations are correctly executed..github/workflows/test.yml (1)
490-525: LGTM! New job addition for PostgreSQL indexer tests.The new job
test-indexer-postgrescorrectly follows the pattern of other test jobs and integrates with SonarCloud. It runs the necessary tests for the PostgreSQL indexer.
indexer/postgres/tests/testdata/init_schema_no_retain_delete.txt
Outdated
Show resolved
Hide resolved
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
|
|
|
|
||
| // CreateTableSql generates a CREATE TABLE statement for the object type. | ||
| func (tm *TableManager) CreateTableSql(writer io.Writer) error { | ||
| _, err := fmt.Fprintf(writer, "CREATE TABLE IF NOT EXISTS %q (\n\t", tm.TableName()) |
There was a problem hiding this comment.
Shall we use ? for avoiding requiring to sanitize table name and module names?
There was a problem hiding this comment.
Can placeholders bind identifiers? I thought it was only parameters. The downside would be that it's hard to inspect the actual SQL in tests and logs
There was a problem hiding this comment.
I see, everything goes to a writer, cannot be a prepared statement indeed. We should probably sanitize the inputs then.
There was a problem hiding this comment.
Schema validation enforces use of the NameFormat regex first, then second %q quotes and escapes these
Description
This PR adds:
cosmossdk.io/indexer/postgresfor the postgres indexercosmossdk.io/indexer/postgres/testsfor postgres indexer integration tests (to keep the dependencies on pgx and embedded postgres out of indexer/postgres)cosmossdk.io/schemamodule schemas with testsAuthor Checklist
All items are required. Please add a note to the item if the item is not applicable and
please add links to any relevant follow up issues.
I have...
!in the type prefix if API or client breaking changeCHANGELOG.mdReviewers Checklist
All items are required. Please add a note if the item is not applicable and please add
your handle next to the items reviewed if you only reviewed selected items.
Please see Pull Request Reviewer section in the contributing guide for more information on how to review a pull request.
I have...
Summary by CodeRabbit
New Features
Documentation