From 50cb391d38c88d556f39ee527b7a0eb8dca4d64e Mon Sep 17 00:00:00 2001 From: Julian Date: Fri, 31 Jan 2025 09:42:31 +0100 Subject: [PATCH 1/4] update main readme --- README.md | 5 ++--- lib/tree_sitter_sql/tree-sitter-sql | 2 +- 2 files changed, 3 insertions(+), 4 deletions(-) diff --git a/README.md b/README.md index ec9e70445..7722071ff 100644 --- a/README.md +++ b/README.md @@ -27,7 +27,7 @@ We plan to support all of the above for SQL and PL/pgSQL function bodies too! Despite the rising popularity of Postgres, support for the PL/pgSQL in IDEs and editors is limited. While there are some _generic_ SQL Language Servers[^1] offering the Postgres syntax as a "flavor" within the parser, they usually fall short due to the ever-evolving and complex syntax of PostgreSQL. There are a few proprietary IDEs[^2] that work well, but the features are only available within the respective IDE. -This Language Server is designed to support Postgres, and only Postgres. The server uses [libpg_query](https://github.com/pganalyze/libpg_query), therefore leveraging the PostgreSQL source to parse the SQL code reliably. Using Postgres within a Language Server might seem unconventional, but it's the only reliable way of parsing all valid PostgreSQL queries. You can find a longer rationale on why This is the Way™ [here](https://pganalyze.com/blog/parse-postgresql-queries-in-ruby). While libpg_query was built to execute SQL, and not to build a language server, any shortcomings have been successfully mitigated in the `parser` crate. You can read the [commented source code](./crates/parser/src/lib.rs) for more details on the inner workings of the parser. +This Language Server is designed to support Postgres, and only Postgres. The server uses [libpg_query](https://github.com/pganalyze/libpg_query), both as a git submodule for access to its protobuf file and as the [pg_query](https://crates.io/crates/pg_query/5.0.0) rust crate, therefore leveraging the PostgreSQL source to parse the SQL code reliably. Using Postgres within a Language Server might seem unconventional, but it's the only reliable way of parsing all valid PostgreSQL queries. You can find a longer rationale on why This is the Way™ [here](https://pganalyze.com/blog/parse-postgresql-queries-in-ruby). While libpg_query was built to execute SQL, and not to build a language server, any shortcomings have been successfully mitigated in the `parser` crate. You can read the [commented source code](./crates/parser/src/lib.rs) for more details on the inner workings of the parser. Once the parser is stable, and a robust and scalable data model is implemented, the language server will not only provide basic features such as semantic highlighting, code completion and syntax error diagnostics, but also serve as the user interface for all the great tooling of the Postgres ecosystem. @@ -86,8 +86,7 @@ The server binary will be installed in `.cargo/bin`. Make sure that `.cargo/bin` ### Github CodeSpaces -Currently, Windows does not support `libpg_query`. You can setup your development environment -on [CodeSpaces](https://github.com/features/codespaces). +You can setup your development environment on [CodeSpaces](https://github.com/features/codespaces). After your codespace boots up, run the following command in the shell to install Rust: diff --git a/lib/tree_sitter_sql/tree-sitter-sql b/lib/tree_sitter_sql/tree-sitter-sql index b9d109588..da2d1eff4 160000 --- a/lib/tree_sitter_sql/tree-sitter-sql +++ b/lib/tree_sitter_sql/tree-sitter-sql @@ -1 +1 @@ -Subproject commit b9d109588d5b5ed986c857464830c2f0bef53f18 +Subproject commit da2d1eff425b146d3c8cab7be8dfa98b11d896dc From 6b87b0bd711b26fc510aa47e47b146b5742f121b Mon Sep 17 00:00:00 2001 From: Julian Date: Fri, 31 Jan 2025 09:58:00 +0100 Subject: [PATCH 2/4] stuff --- crates/pg_lexer/README.md | 8 ++++++++ crates/pg_lexer_codegen/README.md | 7 +++++++ 2 files changed, 15 insertions(+) create mode 100644 crates/pg_lexer/README.md create mode 100644 crates/pg_lexer_codegen/README.md diff --git a/crates/pg_lexer/README.md b/crates/pg_lexer/README.md new file mode 100644 index 000000000..2945d0268 --- /dev/null +++ b/crates/pg_lexer/README.md @@ -0,0 +1,8 @@ +# pg_lexer + +The `pg_lexer` crate exposes the `lex` method, which turns an SQL query text into a `Vec>`: the base for the `pg_parser` and most of pgtools's operations. + +A token is always of a certain `SyntaxKind` kind. That `SyntaxKind` enum is derived from `libpg_query`'s protobuf file. + +The SQL query text is mostly lexed using the `pg_query::scan` method (`pg_query` is just a Rust wrapper around `libpg_query`). +However, that method does not parse required whitespace tokens, so the `lex` method takes care of parsing those and merging them into the result. diff --git a/crates/pg_lexer_codegen/README.md b/crates/pg_lexer_codegen/README.md new file mode 100644 index 000000000..76642e268 --- /dev/null +++ b/crates/pg_lexer_codegen/README.md @@ -0,0 +1,7 @@ +# pg_lexer_codegen + +This crate is responsible for reading `libpg_query`'s protobuf file and turning it into the Rust enum `SyntaxKind`. + +It does so by reading the file from the installed git submodule, parsing it with a protobuf parser, and using a procedural macro to generate the enum. + +Rust requires procedural macros to be defined in a different crate than where they're used, hence this \_codegen crate. From 96d4de53467c960d1e9a7bc7a8f3947199da2f95 Mon Sep 17 00:00:00 2001 From: Julian Date: Fri, 31 Jan 2025 10:00:01 +0100 Subject: [PATCH 3/4] tree-sitter-sql update --- lib/tree_sitter_sql/tree-sitter-sql | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/lib/tree_sitter_sql/tree-sitter-sql b/lib/tree_sitter_sql/tree-sitter-sql index da2d1eff4..302f356b1 160000 --- a/lib/tree_sitter_sql/tree-sitter-sql +++ b/lib/tree_sitter_sql/tree-sitter-sql @@ -1 +1 @@ -Subproject commit da2d1eff425b146d3c8cab7be8dfa98b11d896dc +Subproject commit 302f356b1c832a4d0452d59e061615ccbcaffc7f From e6401d1283645f3c1dacaf5850700a47f0844d2e Mon Sep 17 00:00:00 2001 From: Julian Date: Fri, 31 Jan 2025 10:04:33 +0100 Subject: [PATCH 4/4] ghpages --- lib/tree_sitter_sql/tree-sitter-sql | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/lib/tree_sitter_sql/tree-sitter-sql b/lib/tree_sitter_sql/tree-sitter-sql index 302f356b1..b9d109588 160000 --- a/lib/tree_sitter_sql/tree-sitter-sql +++ b/lib/tree_sitter_sql/tree-sitter-sql @@ -1 +1 @@ -Subproject commit 302f356b1c832a4d0452d59e061615ccbcaffc7f +Subproject commit b9d109588d5b5ed986c857464830c2f0bef53f18