Need additional documentation on high level Crux concepts and architecture

[dgr]

As a new Crux user, I've spent a lot of time reading and re-reading the docs. The following are some suggestions for improvements to the documentation that I believe will make Crux more easily understood by new users.

• 1. DONE (see Docs changes #822 ) An overview of the high-level programming concepts from an API point of view. The current documentation spends time discussing bi-temporality, which is great, but doesn't discuss things like what the node and db concepts are abstracting. You can sort of piece it together from the names, the raw API, and looking at the code, but it would be useful to just have this documented. For instance, when you start a node, what is that doing under the hood? What does a node contain once it has started? When you execute (crux/db node), what is the result of that? What is a DB, exactly? How long can I hold onto it? Why would I want to (or not want to)?

• 2. An overview of how Crux works architecturally and how the abstract architecture maps to each of the various deployment models. For instance, if I'm using the standalone, Kafka, or JDBC topologies, which component is performing which tasks? How does the data flow between the various components in the architecture? Where is it stored? Where is it indexed? When a query happens, which node is running that code?

• 3. Scaling and performance notes. It would be good to understand where the pinch-points in the architecture are and how to work around them. How does Kafka perform versus standalone or JDBC? How much effect does memory have on a node? How much effect does CPU speed have on a node? When should I scale out and add more nodes versus scaling up the nodes I have? Are there any limits to the architecture? Is there any place where the Crux team would say, "Crux is not suited for that type of application or use case" ?

[jarohen]

Incorporated into #386