chore(doc): modify readme and add other docs

CODE_OF_CONDUCT.md

@@ -0,0 +1,40 @@
+## Pledge
+In the interest of fostering an open and welcoming environment, we as contributors and maintainers pledge to make participation in our project and our community a harassment-free experience for everyone, regardless of age, body size, disability, ethnicity, sex characteristics, gender identity and expression, level of experience, education, socio-economic status, nationality, personal appearance, race, religion, or sexual identity and orientation.
+
+## Our Standards
+Examples of behavior that contributes to creating a positive environment include:
+- Using welcoming and inclusive language
+- Being respectful of differing viewpoints and experiences
+- Gracefully accepting constructive criticism
+- Focusing on what is best for the community
+- Showing empathy towards other community members
+- Respecting the work of others by recognizing acknowledgment/citation requests of original authors
+- Being explicit about how we want our own work to be cited or acknowledged
+
+Examples of unacceptable behavior by participants include:
+- The use of sexualized language or imagery and unwelcome sexual attention or advances
+- Sexist, racist, or otherwise exclusionary jokes
+- Trolling, insulting/derogatory comments, and personal or political attacks
+- Public or private harassment
+- Publishing others' private information, such as a physical or electronic address, without explicit permission
+- Other conduct which could reasonably be considered inappropriate in a professional setting
+
+## Our Responsibilities
+Project maintainers are responsible for clarifying the standards of acceptable behavior and are expected to take appropriate and fair corrective action in response to any instances of unacceptable behavior.
+
+Project maintainers have the right and responsibility to remove, edit, or reject comments, commits, code, wiki edits, issues, and other contributions that are not aligned to this Code of Conduct, or to ban temporarily or permanently any contributor for other behaviors that they deem inappropriate, threatening, offensive, or harmful.
+
+## Enforcement
+Instances of abusive, harassing, or otherwise unacceptable behavior may be reported by contacting the project maintainers at AutoMQ. All complaints will be reviewed and investigated and will result in a response that is deemed necessary and appropriate to the circumstances. The project team is obligated to maintain confidentiality with regard to the reporter of an incident. Further details of specific enforcement policies may be posted separately.
+
+Project maintainers who do not follow or enforce the Code of Conduct in good faith may face temporary or permanent repercussions as determined by other members of the project’s leadership.
+
+## Scope
+This Code of Conduct applies both within project spaces and in public spaces when an individual is representing the project or its community. Examples of representing a project or community include using an official project e-mail address, posting via an official social media account, or acting as an appointed representative at an online or offline event. Representation of a project may be further defined and clarified by the Community Standards Committee.
+
+This Code of Conduct is to be understood to apply in addition to any institutional codes of conduct governing individuals working within the project.
+
+## Attribution
+This Code of Conduct is adapted from that of [Airbyte](https://github.com/airbytehq/airbyte), which is adapted from the Contributor Covenant, version 1.4, available at https://www.contributor-covenant.org/version/1/4/code-of-conduct.html
+
+

CONTRIBUTING_GUIDE.md

@@ -0,0 +1,71 @@
+# Contributing to AutoMQ
+
+Thank you for your interest in contributing! We love community contributions.
+Read on to learn how to contribute to AutoMQ.
+We appreciate first time contributors and we are happy to assist you in getting started. In case of questions, just
+reach out to us via [Wechat Group](https://www.automq.com/img/----------------------------1.png)
+or [Slack](https://join.slack.com/t/automq/shared_invite/zt-29h17vye9-thf31ebIVL9oXuRdACnOIA)!
+
+Before getting started, please review AutoMQ's Code of Conduct. Everyone interacting in Slack or Wechat
+follow [Code of Conduct](../community/code-of-conduct.md).
+
+## Code Contributions
+
+Most of the issues that are open for contributions are tagged with `good first issue`.
+Start with
+this [tagged issue](https://github.com/AutoMQ/automq-for-kafka/issues?q=is%3Aissue+is%3Aopen+label%3A%22good+first+issue%22)
+
+The usual workflow of code contribution is:
+
+1. Fork the AutoMQ repository.
+2. Clone the repository locally.
+3. Create a branch for your feature/bug fix with the format `{YOUR_USERNAME}/{FEATURE/BUG}` (
+   e.g. `jdoe/source-stock-api-stream-fix`)
+4. Make and commit changes.
+5. Push your local branch to your fork.
+6. Submit a Pull Request so that we can review your changes.
+7. [Link an existing Issue](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue)
+   that does not include the `needs triage` label to your Pull Request. A pull request without a linked issue will be
+   closed, otherwise.
+8. Write a PR title and description that follows the [Pull Request Handbook](./resources/pull-requests-handbook.md)
+   and [Pull Request Template](https://github.com/airbytehq/airbyte/blob/master/.github/pull_request_template.md).
+9. An AutoMQ maintainer will trigger the CI tests for you and review the code.
+10. Review and respond to feedback and questions by AutoMQ maintainers.
+11. Merge the contribution.
+
+Pull Request reviews are done on a regular basis.
+
+:::info
+Please make sure you respond to our feedback/questions and sign our CLA.
+
+Pull Requests without updates will be closed due inactivity.
+:::
+
+Guidelines to common code contributions:
+
+- [Submit code change to existing Source Connector](change-cdk-connector.md)
+- [Submit a New Connector](submit-new-connector.md)
+
+## Requirement
+
+| Requirement            | Version    |
+|------------------------|------------|
+| Compiling requirements | JDK 17     |
+| Compiling requirements | Scale 2.13 |
+| Running requirements   | JDK 17     |
+
+## Documentation
+
+We welcome Pull Requests that enhance the grammar, structure, or fix typos in our documentation.
+
+## Engage with the Community
+
+Another crucial way to contribute is by reporting bugs and helping other users in the community.
+
+You're welcome to enter
+the [Community Slack](https://join.slack.com/t/automq/shared_invite/zt-29h17vye9-thf31ebIVL9oXuRdACnOIA) and help other
+users or report bugs in Github.
+
+## Attribution
+
+This contributing document is adapted from that of [AutoMQ](https://github.com/airbytehq/airbyte).

README.md

@@ -1,84 +1,108 @@
 <h1 align="center">
-AutoMQ for Kafka
+AutoMQ: Truly serverless Kafka solution that maximizes the benefits of cloud
 </h1>
 <h3 align="center">
-    The truly serverless Kafka solution that maximizes the benefits of cloud
+
 </h3>
 
-[![Docs](https://img.shields.io/badge/Docs-blue)](https://docs.automq.com/zh/docs/automq-s3kafka/YUzOwI7AgiNIgDk1GJAcu6Uanog)
-[![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://www.apache.org/licenses/LICENSE-2.0.html)
+![GitHub License](https://img.shields.io/github/license/AutoMQ/automq-for-kafka)
+![GitHub release (with filter)](https://img.shields.io/github/v/release/AutoMQ/automq-for-kafka)
+![GitHub pull requests](https://img.shields.io/github/issues-pr/AutoMQ/automq-for-kafka)
+![GitHub closed pull requests](https://img.shields.io/github/issues-pr-closed/AutoMQ/automq-for-kafka)
+![GitHub all releases](https://img.shields.io/github/downloads/AutoMQ/automq-for-kafka/total)
+![](https://img.shields.io/badge/Java%20-%20JDK17-green)
+
+---
+[![](https://img.shields.io/badge/official%20document-20B2AA?style=for-the-badge)](https://docs.automq.com/docs/automq-s3kafka/YUzOwI7AgiNIgDk1GJAcu6Uanog)
+&nbsp;
+[![](https://img.shields.io/badge/playground-blue?style=for-the-badge)](https://www.automq.com/home.html)
+&nbsp;
+<a href="https://twitter.com/AutoMQ_Lab"><img src="https://img.shields.io/badge/- @AutoMQ_Lab -424549?style=social&logo=twitter" height=25></a>
+&nbsp;
+<a href="https://www.automq.com/img/----------------------------1.png"><img src="https://img.shields.io/badge/- Wechat -red?style=social&logo=discourse" height=25></a>
+&nbsp;
+
+---
+
+![](https://img.shields.io/badge/aws%20cloud-supported-lightgreen?style=for-the-badge&logo=amazonaws)
+![](https://img.shields.io/badge/google%20cloud-todo-lightyellow?style=for-the-badge&logo=googlecloud)
+![](https://img.shields.io/badge/Azure%20cloud-todo-lightyellow?style=for-the-badge&logo=microsoftazure)
+![](https://img.shields.io/badge/aliyun%20cloud-supported-lightgreen?style=for-the-badge&logo=alibabacloud)
+![](https://img.shields.io/badge/huawei%20cloud-supported-lightgreen?style=for-the-badge&logo=huawei)
+![](https://img.shields.io/badge/baidu%20cloud-supported-lightgreen?style=for-the-badge&logo=baidu)
+![](https://img.shields.io/badge/tencent%20cloud-supported-lightgreen?style=for-the-badge&logo=tencentqq)
+
+
+
+![image](./docs/images/banner-readme.jpeg)
+
+
 
 [//]: # ([![E2E_TEST]&#40;https://github.com/AutoMQ/automq-for-kafka/actions/workflows/nightly-e2e.yml/badge.svg&#41;]&#40;https://github.com/AutoMQ/automq-for-kafka/actions/workflows/nightly-e2e.yml&#41;)
 
-## What is AutoMQ for Kafka
-AutoMQ for Apache Kafka is redesigned based on cloud infrastructure, and users 
-benefit from 
-**elastic computing resources** and nearly **unlimited cheap storage** in 
-the cloud.
+## 🍵What is AutoMQ
+
+AutoMQ is a cloud-native, serverless reinvented Kafka that is easily scalable, manage-less and cost-effective.
+
+
+## 🔶Why AutoMQ
+
+- **Cloud Native**: Built on cloud service. Every system design decision take cloud service's feature and billing items into consideration to offer best low-latency, scalable, reliable and cost-effective Kafka service on cloud.
+- **Serverless**:
+  - Auto Scaling: Watch key metrics of cluster and scale in/out automatically to match you workload and achieve pay-as-you-go.
+  - Scaling in seconds: Computing layer (broker) is stateless and could scale in/out in seconds, which make AutoMQ true serverless. [Learn more](https://docs.automq.com/docs/automq-s3kafka/Eo4Bweg4eiPegykLpAycED1yn7g)
+  - Infinite scalable: Use cloud's object storage as the main storage, never worry about storage capacity.
+- **Manage-less**: Built-in auto-balancer component balance partition and network traffic across brokers automatically. Never worry about partition re-balance. [Learn more](https://docs.automq.com/docs/automq-s3kafka/GSN2wZjeWiR70YkZiRsc6Hqsneh)
+- **Cost effective**: Use object storage as the main storage, take billing items into consideration when design system, fully utilize the cloud service, all of them contribute to AutoMQ and make it 10x cheaper than Apache Kafka. Refer to [this report](https://docs.automq.com/docs/automq-s3kafka/EJBvwM3dNic6uYkZAWwc7nmrnae) to see how we cut Apache Kafka billing by 90% on the cloud.
+- **High performance**: 
+  - Low latency: Use cloud block storage like AWS EBS as the durable cache layer to accelerate write.
+  - High throughput: Use pre-fetching, batch processing and parallel to achieve high throughput.
+  > Refer to the [AutoMQ Performance White Paper](https://docs.automq.com/docs/automq-s3kafka/CYxlwqDBHitThCkxSl2cePxrnBc) to see how we achieve this.
+- **A superior alternative to Apache Kafka**: 100% compatible with Apache Kafka greater than 0.9.x and not lose any good features of it, but cheaper and better.
+
+
+
+## ✨Architecture
 
-It is developed based on Apache Kafka and replace Kafka's local storage with S3 Stream. This design allows Kafka Brokers:
-- Become stateless and could scale up/down in seconds. 
-- By making targeted modifications at the LogSegment level and extensively reusing upper-level code, it guarantees **100% functional compatibility**.
+![image](./docs/images/automq-architecture.png)
 
-Compared to Apache Kafka, AutoMQ for Apache Kafka offers the following advantages:
+AutoMQ use logSegment as a code aspect of Apache Kafka to weave into our features. The architecture including the following main components:
+- **S3Stream**: A streaming library based on object storage offered by AutoMQ. It is the core component of AutoMQ and is responsible for reading and writing data to object storage. [Learn more](https://docs.automq.com/docs/automq-s3kafka/Q8fNwoCDGiBOV6k8CDSccKKRn9d).
+- **Stream**: Stream is an abstraction to mapping the logSegment of Apache Kafka. LogSegment's data, index and other meta will mapping to different type of stream. [Learn more](https://docs.automq.com/docs/automq-s3kafka/GUk7w0ZxniPwN7kUgiicIlHkn9d)
+- **Durable Cache Layer**: AutoMQ use a small size cloud block storage like AWS EBS as the durable cache layer to accelerate write. Pay attention that this is not tiered storage and AutoMQ broker can decoupled from the durable cache layer completely. [Learn more](https://docs.automq.com/docs/automq-s3kafka/X1DBwDdzWiCMmYkglGHcKdjqn9f)
+- **Stream Object**: AutoMQ's data is organized by stream object. Data is read by stream object id through index. One stream have one stream object. [Learn more](https://docs.automq.com/docs/automq-s3kafka/Q8fNwoCDGiBOV6k8CDSccKKRn9d)
+- **Stream set object**: Stream set object is a collection of small stream object that aimed to decrease API invoke times and metadata size. [Learn more](https://docs.automq.com/docs/automq-s3kafka/Q8fNwoCDGiBOV6k8CDSccKKRn9d)
 
-1. Enhanced Scalability: It leverages cloud-native infrastructure and stateless Kafka Brokers, enabling seamless scaling to meet varying workloads. This elasticity allows for efficient resource allocation and ensures optimal performance even during peak periods.
 
-2. 10x Cheaper: By utilizing object storage, it could save storage cost up to 90%; By leveraging serverless architecture and spot instance, it can achieve significant cost savings of up to 90% in compute expenses.
+## ⛄Get started with AutoMQ
 
-3. Simplified Management: No need to manage disks; It automatically performs second-level partition load balancing across Brokers.
+### Run AutoMQ local without cloud
+The easiest way to run AutoMQ. You can experience the feature like fast partition move and network traffic auto-balance. [Learn more](https://docs.automq.com/docs/automq-s3kafka/VKpxwOPvciZmjGkHk5hcTz43nde)
 
-## S3Stream
-AutoMQ for Kafka is built on S3Stream directly, a streaming library based on object storage. Please refer to [S3Stream](https://docs.automq.com/zh/docs/automq-s3kafka/Q8fNwoCDGiBOV6k8CDSccKKRn9d) for more architecture details.
+> Attention: Local mode mock object storage locally and is not a production ready deployment. It is only for demo and test purpose.
 
-AutomMQ for Kafka and AutoMQ for RocketMQ share the same codebase of S3Stream, so please refer to [AutoMQ RocketMQ](https://github.com/AutoMQ/automq-for-rocketmq/tree/main/s3stream) for the source code.
 
-## Quick Start
+### Run AutoMQ on cloud manually
+Deploy AutoMQ manually with released tgz files on cloud, currently compatible with AWS, Aliyun Cloud, Tencent Cloud, Huawei Cloud and Baidu Cloud. [Learn more]( https://docs.automq.com/docs/automq-s3kafka/NBo6wwth3iWUIkkNAbYcPg0mnae)
 
-### Local Run
-#### Launch cluster
-Launch an AutoMQ for Kafka cluster locally using Docker Compose.
+## 💬Community
+You can join the following groups or channels to discuss or ask questions about AutoMQ:
+- Ask questions or report bug by [GitHub Issues](https://github.com/AutoMQ/automq-for-kafka)
+- Discuss about AutoMQ or Kafka by [Slack](https://join.slack.com/t/automq/shared_invite/zt-29h17vye9-thf31ebIVL9oXuRdACnOIA) or [Wechat Group](https://www.automq.com/img/----------------------------1.png)
 
-This cluster comprises 1 Controller node, 2 Broker nodes, and an additional LocalStack container to simulate S3 services locally.
-``` bash
-# launch AutoMQ for Kafka cluster
-docker compose -f docker/docker-compose.yaml up -d
-```
-#### Run a console producer and consumer
-1. Create a topic to store your events:
-``` bash
-# create quickstart-events topic
-bin/kafka-topics.sh --create --topic quickstart-events --bootstrap-server localhost:9094
-```
 
-2. Run the console producer client to write a few events into your topic. By default, each line you enter will result in a separate event being written to the topic.
-``` bash
-bin/kafka-console-producer.sh --topic quickstart-events --bootstrap-server localhost:9094
-```
-You may input some messages like:
-``` text
-This is my first event
-This is my second event
-```
+## 👥How to contribute
+If you've found a problem with AutoMQ, please open a [GitHub Issues](https://github.com/AutoMQ/automq-for-kafka). 
+To contribute to AutoMQ please see [Code of Conduct](CODE_OF_CONDUCT.md) and [Contributing Guide](CONTRIBUTING_GUIDE.md).
+We have a list of [good first issues](https://github.com/AutoMQ/automq-for-kafka/issues?q=is%3Aissue+is%3Aopen+label%3A%22good+first+issue%22) that help you to get started, gain experience, and get familiar with our contribution process.
 
-3. Run the console consumer client to read the events you just created:
-``` bash
-# CRTL-C to exit the consumer
-# run console consumer
-bin/kafka-console-consumer.sh --topic quickstart-events --from-beginning --bootstrap-server localhost:9094
-```
+## 🌈Roadmap
 
-4. Clean up the cluster
-``` bash
-docker compose -f docker/docker-compose.yaml down -v
-```
+Coming soon...
 
-[Explore more](https://docs.automq.com/zh/docs/automq-s3kafka/VKpxwOPvciZmjGkHk5hcTz43nde): Second-level partition migration and automatic traffic rebalancing.
+## ⭐License
+AutoMQ is released under [Apache 2.0 license](https://www.apache.org/licenses/LICENSE-2.0.html). When contributing to AutoMQ, you can find the relevant license header in each file.
 
 
-### AutoMQ Cloud
-Sign up for a [free trial](https://docs.automq.com/zh/docs/automq-s3kafka/EKcdwqXFWixsm0kH5zVcqYzhnle) of AutoMQ Cloud and experience auto scaling with AutoMQ for Kafka.
 
-## License
-[Apache 2.0 license](https://www.apache.org/licenses/LICENSE-2.0.html)