Skip to content

Commit 65cd536

Browse files
lucapettelucapette
authored
docs: revamp README (#1)
1 parent 42f7762 commit 65cd536

File tree

13 files changed

+243
-11
lines changed

13 files changed

+243
-11
lines changed

‎README.md

Lines changed: 116 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -1,29 +1,135 @@
1-
# TypeStream
1+
<div align="center">
2+
<img src="/assets/avatar-transparent.png?raw=true" width="86">
3+
</div>
24

3-
## How to install
5+
<h1 align="center">TypeStream</h1>
6+
7+
<br />
8+
9+
<div align="center">
10+
<a href="https://github.com/typestreamio/typestream/blob/main/LICENSE">
11+
<img src="https://img.shields.io/github/license/typestreamio/typestream" />
12+
</a>
13+
<a href="https://discord.gg/Ha9sJWXb">
14+
<img src="https://img.shields.io/badge/Chat-on%20Discord-blue" alt="Discord invite" />
15+
</a>
16+
</div>
17+
18+
<p align="center">
19+
<a href="#why-typestream">Why TypeStream?</a>
20+
·
21+
<a href="#getting-started">Getting started</a>
22+
·
23+
<a href="#how-to-contribute">How to contribute</a>
24+
·
25+
<a href="#code-of-conduct">Code of conduct</a>
26+
·
27+
<a href="#license">License</a>
28+
</p>
29+
30+
<h3 align="center">
31+
32+
TypeStream is an abstraction layer on top of Kafka that allows you to write
33+
and run <i>typed</i> data pipelines with a minimal, familiar syntax.
34+
35+
</h3 >
36+
37+
## Why TypeStream?
38+
39+
Building streaming data pipelines on top of Kafka comes with some fixed costs.
40+
You have to write the app, test it, then deploy and manage it in production.
41+
Even for the simplest pipelines, this can be a lot of work.
42+
43+
With TypeStream you can write powerful, typed Kafka pipelines like you would
44+
write a simple UNIX pipeline in your terminal. For example, imagine you'd like
45+
to filter a "books" topic. With TypeStream, it's a one liner:
46+
47+
```sh
48+
$ typestream
49+
> cat /dev/kafka/local/topics/books | grep "station" > /dev/kafka/local/topics/stations
50+
```
51+
52+
TypeStream will take care of type-checking your pipeline and then run it for
53+
you. Here's how grepping looks like in action:
54+
55+
![grepping with TypeStream](/assets/vhs/grep.gif?raw=true)
56+
57+
Another common use case that requires a lot of boilerplate is to enrich a topic
58+
with data from an external source. For example, you might have a topic with an
59+
ip address field and you may want to enrich it with country information. With
60+
TypeStream, you can do it (again!) in a single line:
61+
62+
```sh
63+
$ typestream
64+
> cat /dev/kafka/local/topics/page_views | enrich { view -> http "https://api.country.is/#{$view.ip_address}" | cut .country } > /dev/kafka/local/topics/page_views_with_country
65+
```
66+
67+
Here's how enriching looks like in action:
68+
69+
![enriching with TypeStream](/assets/vhs/enrich.gif?raw=true)
70+
71+
As you can see from the previous command, in the spirit of UNIX, we used cut to
72+
extract the country field from the response. Here's the kick with TypeStream,
73+
you can use `cut` (and many other Unix commands) on streams as well:
74+
75+
```sh
76+
$ typestream
77+
> cat /dev/kafka/local/topics/books | cut .title > /dev/kafka/local/topics/book_titles
78+
```
79+
80+
Here's how cutting looks like in action:
81+
82+
![cutting with TypeStream](/assets/vhs/cut.gif?raw=true)
83+
84+
Another problem that TypeStream solves is preventing you from writing faulty pipelines by type-checking them before executing them. Here's type checking in action:
85+
86+
![type checking with TypeStream](/assets/vhs/type-checking.gif?raw=true)
87+
88+
If you'd like to learn more about TypeStream, check out our
89+
[documentation](https://docs.typestream.io/).
90+
91+
## Getting started
492
593
If you use [Homebrew](https://brew.sh/):
694
795
```sh
896
brew install typestreamio/tap/typestream
97+
$ typestream --version
998
```
1099
11-
If you want to build it yourself, refer to our [contributing
12-
guidelines](/CONTRIBUTING.md).
100+
if you see something like this:
101+
102+
```sh
103+
typestream version 2023.08.31+3 42f7762daac1872416bebab7a34d0b79a838d40a (2023-09-02 09:20:52)
104+
```
105+
106+
then you're good to go! You can now run:
107+
108+
```sh
109+
typestream local start
110+
typestream local seed
111+
```
112+
113+
to start a local TypeStream server and seed it with some sample data. Now you're
114+
ready to start writing your own pipelines!
115+
116+
Check out our [documentation](https://docs.typestream.io/) to learn more about
117+
TypeStream.
13118
14119
## How to contribute
15120
16121
We love every form of contribution! Good entry points to the project are:
17122
18-
- Our [contributing guidelines](/CONTRIBUTING.md) document
123+
- Our [contributing guidelines](/CONTRIBUTING.md) document.
19124
- Issues with the tag
20-
[gardening](https://github.com/typestreamio/typestream/issues?q=is%3Aissue+is%3Aopen+label%3Agardening)
125+
[gardening](https://github.com/typestreamio/typestream/issues?q=is%3Aissue+is%3Aopen+label%3Agardening).
21126
- Issues with the tag [good first
22-
patch](https://github.com/typestreamio/typestream/issues?q=is%3Aissue+is%3Aopen+label%3A%22good+first+patch%22)
127+
patch](https://github.com/typestreamio/typestream/issues?q=is%3Aissue+is%3Aopen+label%3A%22good+first+patch%22).
23128
24-
If you're not sure where to start, please open a [new
25-
issue](https://github.com/typestreamio/typestream/issues/new) and we'll gladly help
26-
you get started.
129+
If you're not sure where to start, open a [new
130+
issue](https://github.com/typestreamio/typestream/issues/new) or hop on to our
131+
[discord](https://discord.gg/Ha9sJWXb) server and we'll gladly help you get
132+
started.
27133
28134
## Code of Conduct
29135

‎assets/avatar-transparent.png

22.2 KB
Loading

‎assets/avatar.jpg

33.9 KB
Loading

‎assets/vhs/README.md

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
# Vhs
2+
3+
Here's where we keep the tapes for our gifs. See
4+
[vhs](https://github.com/charmbracelet/vhs) for more information on how this
5+
works. It's a pretty amazing project!
6+
7+
_Note_ the tapes in this directory assume you have a "freshly seeded" local
8+
`TypeStream`.

‎assets/vhs/cut.gif

76.4 KB
Loading

‎assets/vhs/cut.tape

Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,26 @@
1+
Output cut.gif
2+
3+
Set Theme "Monokai Pro"
4+
Set FontSize 22
5+
Set Width 1300
6+
Set Height 650
7+
8+
Type typestream
9+
10+
Enter
11+
12+
Type " cat /dev/kafka/local/topics/books | cut .title"
13+
14+
Enter
15+
16+
Sleep 3s
17+
18+
Ctrl+C
19+
20+
Ctrl+C
21+
22+
Type "# It's that easy 🎉"
23+
24+
Enter
25+
26+
Sleep 3s

‎assets/vhs/enrich.gif

2.31 MB
Loading

‎assets/vhs/enrich.tape

Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,26 @@
1+
Output enrich.gif
2+
3+
Set Theme "Monokai Pro"
4+
Set FontSize 22
5+
Set Width 1300
6+
Set Height 650
7+
8+
Type typestream
9+
10+
Enter
11+
12+
Type ' cat /dev/kafka/local/topics/page_views | enrich { view -> http "https://api.country.is/#{$view.ip_address}" | cut .country }'
13+
14+
Enter
15+
16+
Sleep 5s
17+
18+
Ctrl+C
19+
20+
Ctrl+C
21+
22+
Type "# It's that easy 🎉"
23+
24+
Enter
25+
26+
Sleep 3s

‎assets/vhs/grep.gif

58.3 KB
Loading

‎assets/vhs/grep.tape

Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,26 @@
1+
Output grep.gif
2+
3+
Set Theme "Monokai Pro"
4+
Set FontSize 22
5+
Set Width 1300
6+
Set Height 650
7+
8+
Type typestream
9+
10+
Enter
11+
12+
Type " cat /dev/kafka/local/topics/books | grep 'Station'"
13+
14+
Enter
15+
16+
Sleep 3s
17+
18+
Ctrl+C
19+
20+
Ctrl+C
21+
22+
Type "# It's that easy 🎉"
23+
24+
Enter
25+
26+
Sleep 3s

0 commit comments

Comments
 (0)