Restructured readme and broke out optional info into supplementary topics

arch-notes.md

@@ -0,0 +1,69 @@
+## Library architecture notes
+
+### Interoperability wrappers
+
+Quix Streams base library is developed in C#. We use Interoperability wrappers around <b>C# AoT (Ahead of Time) compiled code</b> to implement support for other languages such as <b>Python</b>. These Interop wrappers are auto-generated using a project called `InteropGenerator` included in the same repository. Ahead-of-time native compilation was a feature introduced officially on .NET 7. Learn more [here](https://learn.microsoft.com/en-us/dotnet/core/deploying/native-aot/).
+
+You can generate these Wrappers again using the `shell scripts` provided for each platform inside the language-specific client. For instance for Python:
+
+- `/src/builds/python/windows/build_native.bat`: Generates Python Interop wrappers for Windows platform.
+- `/src/builds/python/linux/build_native.bat`: Generates Python Interop wrappers for Linux platform.
+- `/src/builds/python/mac/build_native.bat`: Generates Python Interop wrappers for Mac platform.
+
+These scripts compile the C# base library and then use the `InteropGenerator` project to generate the AoT compiled version of the library and the Interop wrappers around that. The result is a structure like this:
+
+```
+
+   ┌───────────────────────────┐
+   │   Python client library   │    /Python/lib/quixstreams
+   └─────────────┬─────────────┘
+                 │
+                 │
+   ┌─────────────▼─────────────┐
+   │  Python Interop wrapper   │    /Python/lib/quixstreams/native/Python  (auto-generated)
+   └─────────────┬─────────────┘
+                 │
+                 │
+   ┌─────────────▼─────────────┐
+   │  C# AoT compiled library  │    /Python/lib/quixstreams/native/win64   (auto-generated)
+   └───────────────────────────┘
+
+```
+
+The non auto-generated `Python client library` still needs to be maintained manually, but this is expected because each language has its own language-specific features and naming conventions that we want to keep aligned with the language user expectations. If you want to add a new feature of the library that is common to all the languages, you should implement that feature in the C# base library first, re-generate the Interop wrappers, and then modify the Python client library to wire up the new feature of the base library.
+
+### Base library
+
+Quix Streams base library is implemented in C#, therefore if your target language is C#, you will use that base library without any [Interoperability wrapper](#interoperability-wrappers) involved on the execution. 
+
+This base library is organized in 3 main layers:
+
+```
+
+   ┌───────────────────────────┐
+   │      Streaming layer      │    /CSharp/QuixStreams.Streaming
+   └─────────────┬─────────────┘
+                 │
+                 │
+   ┌─────────────▼─────────────┐
+   │      Telemetry layer      │    /CSharp/QuixStreams.Telemetry
+   └─────────────┬─────────────┘
+                 │
+                 │
+   ┌─────────────▼─────────────┐
+   │   Kafka Transport layer   │    /CSharp/QuixStreams.Transport.Kafka
+   ├───────────────────────────┤
+   │      Transport layer      │    /CSharp/QuixStreams.Transport
+   └───────────────────────────┘
+
+```
+
+Each layer has his own responsibilities:
+
+- <b>Streaming layer</b>: This is the main layer of the library that users should use by default. It includes all the <b>syntax sugar</b> needed to have a pleasant experience with the library. Another important responsibility of this layer is the <b>embedded time-series buffer</b> system.
+
+- <b>Telemetry layer</b>: This layer is responsible for implementing the `Codecs` serialization and de-serialization for all the <b>Telemetry messages</b> of Quix Streams protocol. This includes time-series and non time-series messages, stream metadata, stream properties messages, parameters definitions, as well as creating the [Stream context](#library-features-) scopes responsible for the separation between data coming from different sources. This layer also implements a `Stream Pipeline` system to concatenate different Stream processes that can be used to implement complex low-level Telemetry services.
+
+- <b>Transport layer</b>: This layer is responsible for the <b>communication with the message broker</b> and implementing some features to deal with the message broker's features and limitations. Some of these features are `message splitting`, `checkpointing`, `partition revocation`, `connectivity issues recovering` among others. This layer is also responsible for implementing a `wrapping messages system` to allow different message types of the library Protocol, and to define the base classes for the `Codecs` implementation of each messages of that Protocol on the upper layers of the library. For <b>Kafka</b> support, this base library uses internally [Confluent .NET Client for Apache Kafka](https://github.com/confluentinc/confluent-kafka-dotnet), which uses the library [librdkafka - the Apache Kafka C/C++ client library](https://github.com/edenhill/librdkafka).
+
+For more information and general questions about the architecture of the library you can join to our official [Slack channel](https://quix.io/slack-invite).

mac-m1-m2-install.md

@@ -0,0 +1,70 @@
+# Installing on Quix Streams on a M1/M2 Mac:
+We are working very hard to support Apple silicon (M1 and M2-based) Macs natively. 
+
+To install Quix Streams on M1/M2 Macs, follow these steps: 
+
+1. To make sure you have Rosetta installed, open Mac Terminal, and run the command `softwareupdate --install-rosetta`.
+
+2. If you don't have Brew installed, install it using the instructions on the [Brew homepage](https://brew.sh). Make sure that after the install script runs, that you perform any configuration recommended by the script.
+
+3. Install an additional terminal, such as iTerm2. You could do this, for example, with `brew install iterm2`.
+
+4. Open finder, go to Applications, and locate iTerm2.
+
+5. Right-click the iTerm2 application icon to display the context-sensitive menu, and select the menu item `Duplicate`.
+
+6. Rename the duplicate created to `iTerm2 rosetta`.
+
+7. Right-click `iTerm2 rosetta` again and select `Get Info`.
+
+8. Tick the `Open using rosetta` checkbox, so that iTerm2 is always opened using Rosetta.
+
+9. Launch `iTerm2 rosetta` by double-clicking it.
+
+10. On the command line, run the `arch` command. This will display `i386`. If not, check your steps so far.
+
+11. Install Brew again. This installs a new copy of Brew to a new directory structure for i386 (x86_64).
+
+12. Open your Zsh profile file, `~/.zprofile`, using a text editor such as Nano. For example, with the command `nano ~/.zprofile`.
+
+13. Add the following code to `~/.zprofile`:
+
+    ```
+    if [ $(arch) = "i386" ]; then
+        PATH="/usr/local/bin/brew:${PATH}"
+        eval "$(/usr/local/bin/brew shellenv)"
+    fi
+    ```
+
+    This will ensure that when you open `iTerm2 rosetta`, your `brew` command will point at the correct (x86_64) Brew installation.
+
+14. Reload your Zsh profile by running `source ~/.zprofile`, or opening a new instance of `iTerm2 rosetta`.
+
+15. Install Python with the command `brew install python3`.
+
+16. Using log messages from `brew`, check where Python was installed, for example: `/usr/local/Cellar/python@3.10/3.10.9/bin/python3`. If not sure, check with `ls /usr/local/Cellar`.
+
+17. Open your `~/.zprofile` file again, and add the following line inside the `if` statement:
+
+    ```
+    if [ $(arch) = "i386" ]; then
+        PATH="/usr/local/Cellar/python@3.10/3.10.9/bin:${PATH}"
+        ...
+    fi
+    ```
+
+18. Reload your Zsh profile by running `source ~/.zprofile`, or by starting a new instance of `iTerm2 rosetta`.
+
+19. Install Quix Streams:
+
+    ```
+    python3 -m pip install quixstreams --user
+    ```
+
+20. You can now run your code that uses Quix Streams:
+
+    ```
+    python3 yourcode.py
+    ```
+
+You have now successfully installed Quix Streams on M1/M2 architectures.