Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ISSUE #1246] Add README.md for pravega connector #1242

Merged
merged 2 commits into from
Sep 9, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 19 additions & 0 deletions eventmesh-connector-plugin/eventmesh-connector-pravega/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
# Pravega Connector
Pravega connector of EventMesh supports taking Pravega as event storage. [Pravega](https://cncf.pravega.io/) is a storage system that exposes Stream as the main primitive for continuous and unbounded data and meets the requirement of EventMesh connector module.

## The Design of Pravega Connector
Pravega has 3 level storage concepts, which are **Event**, **Stream**, **Scope**. An **Event** is represented as a set of bytes within a Stream. A **Stream** is a durable, elastic, append-only, unbounded sequence of bytes having good performance and strong consistency. A **Scope** acts as a namespace for Stream names and all Stream names are unique within their Scope.

Naturally, Pravega connector makes each topic correspond to a pravega stream. Because `EventStreamWriter` and `EventStreamReader` must assign to a special stream in a special scope, each topic corresponds to an `EventStreamWriter` and a `ReaderGroup`(only contains one `EventStreamReader`) as well. What's more, `SubscribeTask` thread will read every topic stream continuously.

**Note**:
- Publish success will respond `-1` as messageId since writing to Pravega will not return message offset of the stream. If you want to use EventMesh with Pravega, please ensure messageId is unimportant.
- Name and ID in Pravega must be `0-9`, `A-Z`, `a-z`, `.`, `-`.
- Topic must be legal in Pravega.
- ReaderGroup name format is `String.format("%s-%s", consumerGroup, topic)`
- Reader name format is `String.format("%s-reader", instanceName).replaceAll("\\(", "-").replaceAll("\\)", "-")`. The `instanceName` consists of `consumerGroup`, config `eventMesh.server.cluster`, EventMesh version and PID.

![pravega-connector](./pravega-connector.jpg)

## Active Pravega Connector
Referring to doc [_Build and Load Plugins_](https://eventmesh.apache.org/docs/installation/runtime#13-build-and-load-plugins) to load `:eventmesh-connector-plugin:eventmesh-connector-pravega` plugin module, and edit `eventMesh.connector.plugin.type=pravega` in config file `eventmesh.properties` to active pravega connector.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.