Need to improve the docs and quick start

[YourTechBud]

The docs are not good enough. People new to the project won't be able to understand its aim and where it fits. Also the quick start seems to be a bit scattered and doesn't paint a clear picture regarding what it does.

[marblewraith]

I think you haven't clearly articulated what it actually is, i mean just reading github alone is a little confusing (comments in parens):

Space Cloud is an open source, high performance web service (mistaken for SaaS/FaaS e.g. algolia?) which provides instant Realtime APIs on the database of your choice. Build Internet Scale apps with the agility of a prototype!

Space Cloud is an open-source, high performance web engine (we've changed terms now... is it a service or an engine? Also engines traditionally are quite computationally intense) which lets you create instant Realtime APIs on most of the databases out there.

Both of these are a bit misleading since [AFAIK] space-cloud is supposed to be a compiled abstraction layer for the stack which allows interaction via REST. You need to work out 2 things IMHO:

1. A succinct description of what the product is.

2. A mission statement regarding what this product hopes to do/provide/cater for.

The docs are not good enough. People new to the project won't be able to understand its aim and where it fits.

You need block diagram(s), indicating how your solution can be integrated in different scenarios, ideally one for single-server and another for multi-server architecture. And be sure to use the existing color scheme for the majority, except in the case of referring other products (e.g. nodeJS would still be green, mysql would still be blue/dolphin, etc).

---

Also i'm curious about the choice of golang. I mean i sort of get it because it facilitates ultra-low latency, but because you dropped the wording:

lets you create instant Realtime APIs

I mean if you're appealing that this should be used in realtime applications, most realtime software in business is critical for operation, that being the case wouldn't you favor erlang or elixir for fault tolerance? Or am i misunderstanding something?

[YourTechBud]

Yes the articulation of the tool is something we need to get right. Block diagrams would definitely help.

Both of these are a bit misleading since [AFAIK] space-cloud is supposed to be a compiled abstraction layer for the stack which allows interaction via REST

@marblewraith you are absolutely right here. To be clear, high performance is for high throughput and low latencies.

Any suggestions on what we can refer to space-cloud as? It's definitely not a service (as in a SaaS tool). Not sure on what to call it. Any ideas?

---

We can have a separate issue opened for why Golang. But to give some idea, golang is an easy to learn language (with an amazing concurrency model) and most of the core cloud technologies are using golang as of date. Elixir was my first choice due to its actor based concurrency model, but we took the decision not to keep such features internal to the platform. Don't want to do all the heavy lifting ourselves. Also the elixir ecosystem is not very rich right now. Golang has client libraries for almost everything. Right now realtime relies on nats or kafka which are already established in the industry. Can always add actor based concurrency to golang later.

Hope that makes sense!

[marblewraith]

Any suggestions on what we can refer to space-cloud as? It's definitely not a service (as in a SaaS tool). Not sure on what to call it. Any ideas?

I'm leaning towards microservice right now, as generically that's what it seems to be. An abstraction based service specifically running for databases within architecture.

Note there is an important distinction between service and microservice, the latter is very granular with regards to purpose and scope, and can be easily integrated with other microservices.

Whether you want to add other descriptors on top of that is up to you, for example *Dedicated* microservice, indicating compiled runtime, independent of other language runtimes.

But i would advise against specifying that the primary output / interface is REST based within the product description, because what happens if in future you want to swap that out for GraphQL or some other standard? Naturally REST will still have to be expounded on, but that will be elsewhere in the docs.

Hope that makes sense!

Yes, i understand the choice of Go now.

[YourTechBud]

I'm leaning towards microservice right now, as generically that's what it seems to be. An abstraction based service specifically running for databases within architecture.

I'm not sure if that's the term we can adopt. Correct me if I'm wrong, but there would be different microservices for newsfeed, comments, user management, etc in a social media app right (depends largely on the architecture)? In Space Cloud you'd have multiple instances of the same process running. In a way it does go back in time to the monolithic services we had. The idea is that a single binary is exposing all the crud functionality. The user, however, can create separate projects for each segment of the app. It wouldn't have major benefits though due to the nature of Space Cloud.

Maybe we can use a more generic term like server? I agree we shouldn't associated strongly with one interface (REST for now) since we do plan on adding more interfaces later on.

To sum it up, how does this sound, Space Cloud is an open source, high performance web server which provides instant Realtime APIs on the database of your choice?

I must add, the functionality of instant APIs will extend way beyond databases. Having said that, does the revised statement seem to be more accurate?

[marblewraith]

Correct me if I'm wrong, but there would be different microservices for newsfeed, comments, user management, etc in a social media app right (depends largely on the architecture)?

You seem to be confusing microservice architecture (MSA; implementation of software) with software architecture (how specific functionality is implemented via software). Granted there is no concrete definition for what a micro service is, however:

"Per Martin Fowler and other experts, (micro)services in a microservice architecture (MSA) are often processes that communicate over a network to fulfill a goal using technology-agnostic protocols such as HTTP.[5][6][7]"

https://en.wikipedia.org/wiki/Microservices

Essentially space cloud, is exposing the database (via REST / HTTP), in other words it's either augmenting or replacing the data access layer (DAL) or the model in typical MVC applications.

Now if you are correct and space cloud is not a microservice (that should be used with other microservices in a server stack), then to prove that, you'd need to demonstrate that the majority of use cases implementing space cloud, would not require any other software in the stack in relation to the database...

Which i don't think you can do, because with the exception of an external API there is practically no use case for directly exposing a database and in all cases (even that of external API's) there are always business logic / application rules that must be programmatically enforced in relation.

To sum it up, how does this sound, Space Cloud is an open source, high performance web server which provides instant Realtime APIs on the database of your choice?

Sounds horrible, because it's not actually a web server.

The connotation of "web server" has a specific meaning for most people and they will have a very clear idea about what to expect that to be i.e. a server on the web that delivers websites or some form of external API.

If you're calling space cloud a web server it conflicts with this (with respects to HTML/websites), but also are you saying there are no cases where it could be used on an intranet?

[YourTechBud]

Now if you are correct and space cloud is not a microservice (that should be used with other microservices in a server stack), then to prove that, you'd need to demonstrate that the majority of use cases implementing space cloud, would not require any other software in the stack in relation to the database...

Yes, this is what I am talking about. Just to be clear, Space Cloud does have external dependencies on softwares like databases, message brokers, object stores, etc. depending on the modules being used. But again they aren't counted as microservices.

I agree we need to build a lot of sample applications to demonstrate that this is actually possible. Google Firebase has made life for us a lot easier in this regard though. Curious, what use cases would you like count as valid demonstrations?

Which i don't think you can do, because with the exception of an external API there is practically no use case for directly exposing a database and in all cases (even that of external API's) there are always business logic / application rules that must be programmatically enforced in relation.

Talking about application rules in terms of authorisation, space cloud takes care of that. Take my word for it. And I am not talking about trivial rules like role based authentication. We can even enforce rules which rely on policies stored in the database by application logic. For example, in Instagram, you can view someone's post only if you follow them or they are a public account. This technically does fall under the bracket of authorisation and space cloud can take care of it without writing any code. We can have a separate thread opened up about the the security considerations needed to be taken in such a project. One of the goals of space cloud is to make security a default and would love any contribution on that regard.

Regarding business logic. Yes you might need to write your own business logic. Space cloud offers a module called functions wherein a user can write functions and run them as microservices (this isn't the traditional FaaS you might know). But again in this case, space cloud would be acting as a gateway and a DAL(if required. This is an intranet use case btw). All function invocations (trigger requests) would be routed via space cloud. In most use cases space cloud will be consumed via the clients directly. But there is nothing stopping you to use it as an internal DAL only.

The connotation of "web server" has a specific meaning for most people and they will have a very clear idea about what to expect that to be i.e. a server on the web that delivers websites or some form of external API.

What if I say it can serve static assets using it's file storage module backed by amazon s3 or the local file system? The point of space cloud is to create instant APIs for external use right?

Also, before things get heated up, I might be missing out on some vital information. Such a tool is unheard of (I wouldn't risk saying unique) and does go against conventional best practises. I'm still trying to figure out what's the language I need to speak to convey the idea. Apologies if I make any mistakes. And you are helping me a lot.

[marblewraith]

Things are getting clearer, seems i had the wrong idea, because you described it on reddit only with relation to the database.

But the scope seems to be larger, like you're trying to create your own back-end ecosystem, almost like an open source orchestration tool + inhouse runtime.

I'll need to think on it more.

[YourTechBud]

I wouldn't be that ambitious to call it an entire ecosystem right now (that would make the scope a bit too large).

But the objective is to rely on established pieces of technology for the various tasks (kafka for realtime, nats for functions, kubernetes for orchestration).

Your questions do add a lot of value. I'm working on your feedback to prevent such misunderstandings in the future. Thanks a ton buddy!

But do help me with a one liner which can explain what this project does.

[YourTechBud]

@marblewraith I've updated the readme. Do you think it's more accurate now?

[marblewraith]

Definitely an improvement on the docs, i was thinking hard on it over the past few weeks and i can't really come up with anything better, so perhaps you're right and sticking with a generic term such as "web server" is the best option for now.

Though i did have a thought, if you want to continue the "space" analogy, perhaps it's worth breaking the project up into smaller repo's so naming is easier? For example

• Runtime implementation: spaceuptech/space-station (which has "docks" for interfaces to other modules)
• Configuration / admin / orchestration: spaceuptech/mission-control

[YourTechBud]

@marblewraith I hear your point. Love the idea to take the "space" analogy forward. I really want to do it right way. Just thinking how to go forward with it.

I'm more inclined towards the mono repo approach to keep everything in a single place though. There would be some other way to make this happen.

[YourTechBud]

The docs are pretty robust now!