digoal
2023-12-14
PostgreSQL , PolarDB , DuckDB , rust , postgresql wire protocol
https://github.com/sunng87/pgwire
PostgreSQL wire protocol implemented as a rust library.
适合需要编写兼容PostgreSQL交互协议的服务的场景. 例如cockroach数据库的交互协议.
Build Postgres compatible access layer for your data service.
This library implements PostgreSQL Wire Protocol, and provide essential APIs to
write PostgreSQL comptible servers and clients.
This library is a work in progress and in its relatively early stage. There is
no guarantee for API stability. I'm constantly introducing break changes during
this period. And also sorry for lack of API docs, examples may get you familiar
with its usage.
- Message format
- Frontend-Backend protocol messages
- Logical replication streaming protocol message
- Backend TCP/TLS server on Tokio
- Frontend-Backend interaction over TCP
- SSL Request and Response
- Startup
- No authentication
- Clear-text password authentication
- Md5 Password authentication
- SASL SCRAM authentication
- SCRAM-SHA-256
- SCRAM-SHA-256-PLUS
- Simple Query and Response
- Extended Query and Response
- Parse
- Bind
- Execute
- Describe
- Sync
- Termination
- Cancel
- Error and Notice
- Copy
- Notification
- Logical replication over TCP
- APIs
- Startup APIs
- AuthSource API, fetching and hashing passwords
- Server parameters API, ready but not very good
- Simple Query API
- Extended Query API
- QueryParser API, for transforming prepared statement
- PortalStore API, for caching statements and portals
- ResultSet builder/encoder API
- Query Cancellation API
- Error and Notice API
- Copy API
- Copy-in
- Copy-out
- Copy-both
- Logical replication server API
- Startup APIs
Postgres Wire Protocol is a relatively general-purpose Layer-7 protocol. There
are 3 parts of the protocol:
- Startup: client-server handshake and authentication.
- Simple Query: The text-based query protocol of postgresql. Query are provided
as string, and server is allowed to stream data in response. - Extended Query: A new sub-protocol for query which has ability to cache the
query on server-side and reuse it with new parameters. The response part is
identical to Simple Query.
Also note that Postgres Wire Protocol has no semantics about SQL, so literally
you can use any query language, data formats or even natural language to
interact with the backend.
The response are always encoded as data row format. And there is a field
description as header of the data to describe its name, type and format.
To use pgwire in your server application, you will need to implement two key
components: startup processor and query processor. For query
processing, there are two kinds of queries: simple and extended. By adding
SimpleQueryHandler to your application, you will get psql command-line tool
compatibility. And for more language drivers and additional prepared statement,
binary encoding support, ExtendedQueryHandler is required.
Examples are provided to demo the very basic usage of pgwire on server side:
examples/sqlite.rs: uses an in-memory sqlite database at its core and serves
it with postgresql protocol. This is a full example with both simple and
extended query implementation.examples/gluesql.rs: uses an in-memory
gluesql at its core and serves
it with postgresql protocol.examples/server.rs: demos a server that always returns fixed results.examples/secure_server.rs: demos a server with ssl support and always
returns fixed results.examples/scram.rs: demos how to configure more secure authentication
mechanism:
SCRAMexamples/datafusion.rs: demos a postgres compatible server backed by
datafusion query engine. This example allows you toLOADcsv files as
datafusion table and runSELECTqueries on them.
I think in most case you do not need pgwire to build a postgresql client,
existing postgresql client like
rust-postgres should fit your
scenarios. Please rise an issue if there is a scenario.
- GreptimeDB: Cloud-native
time-series database - sqld: A server frontend for
libSQL - risinglight: OLAP database
system for educational purpose - PeerDB Postgres first ETL/ELT, enabling
10x faster data movement in and out of Postgres - CeresDB CeresDB is a high-performance,
distributed, cloud native time-series database from AntGroup.
This library is released under MIT/Apache dual license.
