Roadmap for demo 1

[huachaohuang]

The project is just started and is still in the demo stage now.
The primary goal in this stage is to explore the possibility of our designs.

We plan to release the first demo at the end of Sep 2021.
We are going to achieve the following objectives:

• A working demo with simple read/write APIs on AWS
• A demo report about what we did and the lessons learned

Tasks:

• Cache
• HybridStorage
• SstableStorage
• ParquetStorage
• RemoteJournal
• RemoteManifest
• RemoteCompaction
• Support S3 file system and SELECT
• A purpose-built benchmark tool

However, we are short of hands.
If you think we are behind schedule, please push us with a 👍

Update: this demo has ended, please check the report for more details.

[huachaohuang]

Congratulations! We have finished this demo. Thank you for watching and participating in it!
For more details, please check the report.
We will keep this issue open for a few days and welcome you to discuss the demo with us here.

[hi-wanghan]

Congrats! What's your plan on the networking stuff? Good news is the industry wholesale transit prices have fallen at least 50% during the past 8 years <though AWS is never part of it ;)>

[huachaohuang]

We may need some optimizations in the design and implementation.
Networking is not a blocker so far. It's only one piece of the big picture.

[tisonkun]

@huachaohuang you can link the post of demo 1 in the description https://engula.com/posts/demo-1/ .

Congrats! I'm reading it.

[tisonkun]

The first question I'd like to ask is that how can I reproduce your demo?

You can omit details independent to Engula, like assuming the reader already sets up two r6gd.2xlarge instances, but elaborate how to setup the storage engine cluster with your command-line tools. And I guess it requires setting up a Prometheus service also.

Although I can read the report literally, a reproducible steps (could be an appendix to the original report) is interesting to explore. Depend on your schedule and whether retain the history generating the report, you may consider this suggestion.

[huachaohuang]

@tisonkun Hmm, let me document it somewhere. But I have to say it is cumbersome since I haven't pay attention to the deployment stuff yet.

[huachaohuang]

I added some guides to run experiments on README.
It is too hard to explain every detail to reproduce the report, but I think it's good enough to get people started on their experiments, have fun :)

[huachaohuang]

We are closing this issue now, thanks for watching and participating in this demo.
The roadmap for the next demo is here.